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Abstract 

Accurately  modeling  the  ionosphere  is  crucial  to  forecasting  potential  operational 
impacts,  including  spacecraft  charging,  radio  communication  blackout,  navigational 
errors,  and  safety  issues  for  astronauts.  The  Floating  Potential  Measurement  Unit 
(FPMU)  aboard  the  International  Space  Station  (ISS)  monitors  the  ISS  charging  en¬ 
vironment  and  provides  a  unique  electron  density  and  electron  temperature  in-situ 
data  source  just  above  the  F2  peak  for  the  purpose  of  ionospheric  model  valida¬ 
tion.  Since  FPMU  data  availability  is  limited,  an  ionospheric  model  that  accurately 
predicts  the  ISS  plasma  environment  is  desirable  for  ISS  mission  planning  and  situ¬ 
ational  awareness  when  FPMU  data  is  unavailable.  Electron  densities  and  tempera¬ 
tures  from  four  ionospheric  models  (International  Reference  Ionosphere  [IRI],  Coupled 
Thermosphere  Ionosphere  Plasmasphere  Electrodynamics  model  [CTIPe] ,  Ionospheric 
Forecast  Model  [IFM],  and  Global  Assimilation  of  Ionospheric  Measurements  model 
[GAIM])  were  compared  to  in-situ  FPMU  values  across  a  range  of  geomagnetic  and 
solar  conditions.  The  climatological  and  assimilative  models  (IRI  and  GAIM)  per¬ 
formed  the  best  overall  across  all  conditions,  while  the  pure  physics  based  models 
(IFM  and  CTIPe)  struggled  the  most  to  accurately  predict  the  ISS  plasma  environ¬ 
ment.  The  models  struggled  most  during  severe  solar  minimum  conditions  and  periods 
of  moderate  geomagnetic  and  high  solar  activity.  IRI-2012  and  GAIM  represent  the 
best  candidates  for  use  by  NASA  as  an  ISS  mission  planning  tool. 
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A  COMPARISON  OF  IONOSPHERIC  MODEL  PERFORMANCE  FOR 
INTERNATIONAL  SPACE  STATION  ORBITS 

I.  Introduction 

1.1  Motivation  and  Background 

As  our  reliance  on  satellites,  Global  Positioning  System  (GPS)  navigation,  and 
long  range  communication  increases,  it  has  become  critical  to  better  understand  the 
environment  our  satellites  and  radio  signals  operate  in  and  through,  known  as  the 
ionosphere.  The  ionosphere  is  the  near  earth  subset  of  space  located  roughly  between 
90  km  and  1500km  and  its  conditions  can  fluctuate  depending  on  geomagnetic  storm¬ 
ing  conditions,  time  of  day,  seasons,  solar  cycle,  and  solar  activity.  These  events  can 
cause  radio  communication  blackout,  navigational  errors,  spacecraft  charging,  safety 
issues  for  humans  in  space,  and  other  hazards.  Accurate  ionospheric  modeling  is 
crucial  to  help  mitigate  and  predict  these  threats. 

1.2  Impact  of  the  Ionosphere  on  the  International  Space  Station 

The  International  Space  Station  (ISS)  operates  at  an  altitude  of  400km,  above 
the  peak  plasma  density  height  of  the  ionosphere  (typically  around  300km);  the  area 
above  this  peak  is  known  as  the  topside  ionosphere.  The  ISS  has  many  missions, 
some  of  which  involve  Extra  Vehicular  Activities  (EVAs),  or  spacewalks,  to  perform 
repairs  to  the  ISS  and/or  experiments.  Due  to  safety  concerns  from  charging,  NASA 
has  identified  a  risk  to  astronauts  during  EVAs  if  the  ISS  charges  to  more  than  40 
volts  ( Ferguson  et  al,  2003).  Charging  of  the  ISS  can  be  especially  significant  when 
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it  passes  through  areas  of  cold,  high-density  plasma;  this  can  lead  to  high  potential 
differences  between  the  ISS  and  the  surrounding  plasma  and  lead  to  arcing. 

The  ISS  measures  the  surrounding  ionosphere  with  an  instrument  called  the  Float¬ 
ing  Potential  Measurement  Unit  (FPMU).  The  FPMU  was  installed  because  the  ISS 
faces  unique  charging  situations;  not  only  do  astronauts  on  board  need  to  perform 
EVAs,  but  also  the  ISS  solar  panels  are  much  larger  and  operate  at  a  much  higher  volt¬ 
age  than  most  spacecraft  ( Carruth  et  al,  2001).  The  exposed  interconnects  between 
the  solar  panels  can  accumulate  ions  as  the  ISS  travels  through  the  ionosphere,  creat¬ 
ing  an  electrical  potential  difference  with  the  surrounding  plasma  (known  as  floating 
potential),  causing  charging.  While  one  of  the  functions  of  the  FPMU  is  to  measure 
the  surrounding  ionosphere  to  try  to  understand  why  the  ISS  charges,  its  primary 
purpose  is  to  monitor  charging  levels  of  the  ISS  and  provide  data  that  can  be  used 
to  validate  the  ISS  charging  models  ( Barjatya  et  al.,  2009).  As  a  result,  the  FPMU 
is  not  operated  continuously;  it  is  activated  by  commands  from  the  ground  and  only 
records  data  for  specific  time  durations  when  ISS  charge  monitoring  is  deemed  most 
necessary  (i.e.,  prior  to  an  EVA,  during  an  ISS  configuration  change,  etc.). 

1.3  Ionospheric  Modeling 

Various  computer-based  models  have  been  created  to  help  predict  densities  and 
temperatures  throughout  the  ionosphere.  Many  validation  efforts  for  these  models 
have  taken  place  below  the  peak  plasma  density  of  the  ionosphere  due  to  the  ability  of 
using  ionosondes  to  determine  plasma  densities  at  various  altitudes  ( Tascione ,  2010). 
Similarly,  total  electron  content  (TEC)  measurements  have  been  taken  using  signals 
from  various  satellites  (mostly  GPS)  that  travel  through  the  ionosphere.  However, 
limited  work  has  been  done  to  validate  these  models  for  the  topside  ionosphere  due 
to  the  difficulty  in  obtaining  plasma  density  data  in  that  region,  since  radio  signals 
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either  pass  through  the  ionosphere  or  are  reflected  before  they  reach  the  peak  plasma 
density.  While  signals  that  pass  through  the  ionosphere  enable  the  calculation  of  TEC 
values,  reflected  signals  are  critical  to  determining  electron  density  values  at  specific 
heights,  since  reflection  is  driven  by  the  plasma  frequency  in  a  certain  region,  which  is 
directly  related  to  electron  density.  The  heights  associated  with  those  density  values 
are  determined  by  the  time  it  takes  for  a  signal  to  be  reflected  back.  As  a  result,  most 
models  use  plasma  density  values  from  the  bottom-side  ionosphere,  TEC  values,  and 
electron  transport  models  to  extrapolate  plasma  densities  for  the  topside  ionosphere 
(. Reinisch  and  Huang ,  2001). 

1.4  Research  Focus 

This  research  effort  aims  to  answer  two  main  questions:  How  well  do  the  current 
ionospheric  models  perform  in  the  topside  ionosphere  just  above  the  F2  peak?  Are 
the  current  ionospheric  models  accurate  enough  at  ISS  orbits  for  NASA  to  use  them 
for  mission  planning  and  to  predict  electron  densities  and  temperatures  when  the 
FPMU  is  unable  to  collect  data? 

1.5  Document  Organization 

The  remainder  of  this  document  is  organized  as  follows:  in  Chapter  II,  theory 
related  to  the  ionosphere  is  explained,  along  with  a  brief  description  of  each  of  the 
ionospheric  models  used  in  this  effort.  Chapter  III  describes  the  research  methodology 
used  in  this  effort.  Next,  analysis  results  are  presented  in  Chapter  IV.  Finally,  a 
summary  of  the  results  and  recommendations  for  future  work  are  given  in  Chapter 
V. 
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II.  Background 


This  chapter  provides  background  information  about  the  ionosphere,  the  iono¬ 
spheric  models  considered  in  this  effort,  and  the  data  source  on  the  International 
Space  Station  used  for  validation. 

2.1  The  Ionosphere 

The  ionosphere  is  the  region  of  Earth’s  atmosphere  located  from  90  km  to  1500 
km  above  Earth  that  contains  a  large  amount  of  ions  and  electrons  formed  primarily 
from  photoionization  from  extreme  ultraviolet  and  soft  x-ray  radiation  from  the  Sun. 
These  ions  and  electrons  are  subject  to  many  different  processes,  including  chemical 
reactions,  diffusion,  recombination,  and  transport  by  neutral  winds.  The  ions  and 
electrons  form  a  plasma,  which  is  a  population  of  ionized  (e.g.,  charged)  particles 
that  are  quasi-neutral  as  a  whole  and  exhibit  collective  behavior.  This  plasma  is 
very  important  because  it  can  create  charging  hazards  for  spacecraft  and  affect  signal 
propagation  through  the  ionosphere,  potentially  affecting  all  satellite  and  certain  radio 
communications. 

The  main  source  of  ionization  in  the  ionosphere  is  photoionization  caused  by  solar 
radiation.  While  solar  irradiance  increases  with  altitude,  densities  of  the  neutral 
particles  that  become  ionized  by  this  solar  radiation  decrease  with  height;  this  creates 
a  peak  in  ionization  known  as  a  Chapman  layer  (Figure  1).  Chapman  layers,  combined 
with  chemical  ion/electron  production  and  loss  processes,  diffusion,  and  transport 
mechanisms,  create  several  distinct  layers  (D,  E,  and  F)  in  the  ionosphere,  with 
different  processes  dominating  in  each  layer.  Chemical  processes  dominate  in  the 
D  (60-100  km)  and  E  (100-150  km)  regions,  where  neutral  densities  are  greater. 
Typical  ion/elcctron  densities  in  these  regions  are  on  the  order  of  1011  m~3.  Transport 
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Figure  1.  Chapman  layer  and  ionization  curve  (dashed  line).  (Adapted  from  Steadman 
2011) 

processes  have  a  larger  impact  in  the  F  region,  which  is  split  further  into  two  regions: 
F\  (150-250km)  and  F2  (250-600km).  These  transport  processes  lead  to  larger  electron 
densities,  and  increase  until  they  reach  the  F2  region,  where  the  transport  processes 
dominate.  A  distinct  peak  is  formed  in  the  F2  region  where  the  plasma  transport 
mechanisms  balance  the  chemical  loss  processes,  typically  on  the  order  of  1012  m~3 
( Schunk  and  Nagy ,  2009).  The  region  above  the  F2  peak  is  generally  known  as  the 
topside  ionosphere.  Figure  2  shows  a  vertical  profile  of  the  distinct  layers. 

Typical  ionospheric  parameters  include  electron  density  ( Ne ),  ion  density  (TVj), 
electron  temperature  (Te),  total  electron  content  (TEC),  density  of  the  F2  peak 
( NmF2 ),  and  the  height  of  the  F2  peak  ( hmF2 ).  TEC  is  the  total  integrated  number 
of  electrons  between  two  points,  usually  measured  vertically.  It  is  one  of  the  most 
common  parameters  used  for  the  comparison  of  ionospheric  models  due  to  the  ability 
of  GPS  satellites  to  measure  TEC  from  distortions  in  transmitted  signals.  Due  to 
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Figure  2.  Vertical  profile  of  the  ionosphere.  (Adapted  from  Banks  et  al.,  1976) 

the  sensors  incorporated  into  the  FPMU,  this  effort  will  focus  on  electron  density 
and  electron  temperature.  Model-predicted  TEC  will  also  be  considered  as  a  means 
to  further  compare  model  differences  among  electron  densities  and  observe  overall 
trends. 

2.1.1  Electron  Density  Anomalies. 

While  there  are  many  different  electron  density  anomalies,  two  of  the  most  com¬ 
mon  are  the  Appleton  Anomaly  and  plasma  bubbles.  The  Appleton  anomaly  occurs 
in  equatorial  regions  during  the  daytime.  It  is  caused  by  eastward  ionospheric  cur¬ 
rents  generated  by  the  neutral  wind  that  drive  an  upward  plasma  flow,  which  then 
diffuses  down  magnetic  held  lines  due  to  gravity.  It  creates  a  fountain-like  effect,  with 
enhancement  in  plasma  densities  on  either  side  of  the  magnetic  equator  ( Schunk  and 
Nagy ,  2009).  Plasma  bubbles  occur  primarily  at  night,  also  near  the  magnetic  equator 
and  partially  due  to  the  same  processes  that  create  the  Appleton  anomaly.  Around 
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dusk,  there  is  an  enhancement  in  the  eastward  electric  held  from  increased  neutral 
winds  at  dusk  and  conductivity  gradients  across  the  terminator,  sometimes  known 
as  pre-reversal  enhancement.  This  increased  electric  held  again  drives  an  upward 
plasma  how;  however,  the  local  ionosphere  then  rotates  into  the  night  sector,  where 
production  from  sunlight  ceases  and  chemical  loss  processes  dominate,  causing  a  swift 
decrease  in  electron  densities  in  the  F\  layer.  This  in  turn  creates  what  is  known  as 
a  Rayleigh- Taylor  instability,  where  a  high-density  fluid  is  above  a  low  density  fluid. 
Once  a  perturbation  in  the  boundary  layer  is  created  (sometimes  by  gravity  waves), 
magnetic  field-aligned  electron  density  depletions  can  then  grow  rapidly  into  plasma 
bubbles.  They  can  reach  heights  of  1500  km,  grow  as  quickly  as  5  km  s-1,  and  extend 
horizontally  for  several  thousand  kilometers  ( Schunk  and  Nagy,  2009). 

2.1.2  Electron  Temperature. 

The  electron  temperature  is  an  indication  of  how  much  energy  is  present  in  a 
plasma.  The  electron  energy  is  important  because  it  determines  what  kinds  of  colli¬ 
sions  and  reactions  can  take  place,  which  drive  the  creation  of  ions,  electrons,  and  the 
ionosphere  in  general.  Additionally,  electron  density  and  electron  temperature  tend 
to  be  inversely  correlated,  because  electron  cooling  rates  are  higher  when  electron 
densities  are  greater  ( Schunk  and  Nagy ,  2009). 

2.2  Ionospheric  Models 

While  many  different  models  exist  for  the  ionosphere,  four  will  be  considered  in 
this  effort,  with  multiple  versions  of  certain  models  compared  in  some  cases:  Interna¬ 
tional  Reference  Ionosphere  (IRI),  Coupled  Thermosphere  Ionosphere  Plasmasphere 
Electrodynamics  (CTIPe),  Ionospheric  Forecast  Model  (IFM),  and  Global  Assimila¬ 
tion  of  Ionospheric  Measurements  (GAIM).  Each  model  takes  a  different  approach 
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to  modeling  the  ionosphere  and  is  well  established,  providing  a  good  sample  set  of 
ionospheric  models  against  which  FPMU  data  is  compared.  The  GAIM  model  holds 
special  interest  from  an  Air  Force  perspective,  since  it  is  used  operationally  by  the 
Air  Force  Weather  Agency  to  help  predict  the  operational  impact  of  conditions  in  the 
ionosphere. 

Each  ionospheric  model  is  briefly  described  here,  since  a  detailed  description  of 
each  would  be  impractical  for  this  effort.  Descriptions  includes  model  inputs,  outputs, 
and  availability,  as  well  as  references  for  more  in-depth  information  about  each  model 
for  the  reader. 

2.2.1  International  Reference  Ionosphere. 

IRI  (International  Reference  Ionosphere)  is  an  empirical  computer-based  iono¬ 
spheric  model  sponsored  by  the  Committee  on  Space  Research  (COSPAR)  and  the 
International  Union  of  Radio  Science  (URSI).  It  is  based  on  a  wide  range  of  ground 
and  space  data  from  a  worldwide  network  of  ionosondes,  as  well  as  in  situ  instru¬ 
ments  on  many  satellites  and  rockets  ( Bilitza  and  Remised,  2008).  Because  many  of 
these  data  sources  are  at  mid-latitudes,  it  tends  to  perform  better  there  than  at  low 
latitudes  (Bilitza  et  al. ,  2011).  IRI  is  considered  the  international  standard  for  the 
ionosphere  by  the  International  Organization  for  Standardization  (ISO). 

Since  IRI  is  an  empirical  model,  it  is  updated  regularly  as  new  measurement  data 
emerges  and  provides  a  climatological  reference  to  compare  to  forecasts  from  other 
models.  IRI-90  was  the  first  version  of  the  IRI  model  to  be  released  as  a  computer 
model.  IRI-2012  is  the  most  recent  version  of  the  IRI  model.  Since  it  was  released 
recently,  there  are  very  few  studies  that  have  used  the  model.  However,  IRI-2007 
(the  major  version  prior  to  IRI-12)  has  been  used  extensively.  IRI- 12  includes  a  new 
WmF2model  and  topside  electron  density  model,  along  with  updated  averages  ( Bilitza 


et  al,  2011). 

Required  inputs  for  IRI-1990  and  IRI-2012  include  latitude,  longitude,  height,  3- 
hour  Ap  magnetic  index,  as  well  as  daily,  365-day  averaged,  and  810-day  averaged 
F10.7  index  values.  IRI  outputs  include  electron  temperature,  electron  density,  ion 
drift,  ion  composition  (0+,  H+,  N+,  He+,  O^,  NO+ ,  and  Cluster +),  and  ion  tem¬ 
perature,  all  from  50-1500km  ( Bilitza  and  Reinisch,  2008).  IRI  is  the  only  model 
compared  in  this  effort  that  does  not  produce  three-dimensional  output.  All  major 
versions  of  IRI  (including  those  used  in  this  effort)  are  available  for  download  as 
FORTRAN  code  at  http://iri.gsfc.nasa.gov/. 

2.2.2  Coupled  Thermosphere  Ionosphere  Plasmasphere  Electrodynam¬ 
ics  Model. 

The  Coupled  Thermosphere  Ionosphere  Plasmasphere  Electrodynamics  (CTIPe) 
model  is  a  global,  three-dimensional,  time-dependent  physics-based  model  that  in¬ 
corporates  models  of  four  distinct  components  ( Codrescu  et  al.,  2012)  These  in¬ 
clude  a  global  thermosphere  model,  a  high-latitude  ionosphere  model,  a  mid-  and 
low- latitude  ionosphere/plasmasphere  model,  and  an  electrodynamical  calculation  of 
the  global  dynamo  electric  held.  The  CTIPe  model  runs  each  of  these  components 
concurrently  and  couples  them  with  respect  to  energy,  momentum,  and  continuity 
( Codrescu  et  al,  2012).  The  CTIPe  model  is  currently  used  by  NOAA’s  Space 
Weather  Prediction  Center  to  model  the  ionosphere.  Additionally,  CTIPe  can  be 
run  online  through  NASA’s  Community  Coordinated  Modeling  Center  (CCMC)  at 
http://ccmc.gsfc.  nasa.gov/models/modelinfo.php?model=CTIPe. 

Required  inputs  include  F10.7  index  values  (daily  F10.7  and  90-day  average  F10.7), 
magnitude  of  the  magnetic  held,  IMF  clock  angle,  dipole  tilt,  solar  wind  speed,  solar 
wind  density,  Hemispheric  Power  index  value,  and  Hemispheric  power  in  gigawatts. 
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CTIPe  has  a  2°  latitude  resolution,  an  18°  longitude  resolution,  with  vertical  spacing 
broken  up  into  15  pressure  levels  at  a  resolution  of  one  scale  height.  Outputs  include 
neutral  densities  of  02,  O  and  N2,  electron  densities,  and  ion  densities  (0+,  H+, 
N+,  O2,  N2)  from  140km  to  2000km.  CTIPe  can  also  calculate  ion  and  electron 
temperatures,  but  only  within  geomagnetic  latitudes  of  ±  50°. 

2.2.3  Ionospheric  Forecast  Model. 

The  Ionospheric  Forecast  Model  (IFM)  is  a  physics-based  numerical  solution  of 
the  global  ionosphere  using  plasma  continuity,  momentum  and  energy  equations  ( Zhu 
et  al,  2006).  It  was  developed  by  Utah  State  University  (USU)  and  can  provide  a 
24-hour  forecast  of  the  ionosphere  in  30  minute  increments.  The  number  of  inputs 
required  for  IFM  is  actually  less  than  IRI;  IFM  only  requires  the  date,  run  dura¬ 
tion,  F10.7  index  values,  and  Kp  index  values.  It  creates  a  three-dimensional,  time- 
dependent  density  distribution  of  electrons  and  the  major  ion  constituents  (H+,  O+, 
NO+,  and  02+)  in  the  ionosphere,  as  well  as  electron  and  ion  temperatures.  IFM’s 
3-D  grid  has  a  resolution  of  3°  latitude  and  7°  longitude.  Vertically,  output  spans 
from  90-1600km,  and  vertical  resolution  is  4km  in  the  E  region  and  20  km  in  the  F 
region  ( Scherliess  et  al.,  2006).  IFM  calculates  electron  temperatures  using  two  dif¬ 
ferent  models  ( Schunk ,  2012).  Within  ±  45°  of  the  geomagnetic  equator,  Titheridge’s 
empirical  electron  temperature  model  is  used  ( Titheridge ,  1998).  Above  ±  20°  geo¬ 
magnetic  latitudes,  electron  temperature  is  calculated  from  a  numerical  solution  to 
the  continuity,  momentum,  and  energy  equation  ( Schunk  et  al. ,  1986).  Then,  the  elec¬ 
tron  temperature  values  from  both  models  are  blended  from  ±  25°  to  ±  40°  magnetic 
latitude.  Electron  temperature  values  are  then  used  by  IFM  to  calculate  electron 
density  values. 
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2.2.4  Global  Assimilation  of  Ionospheric  Measurements  Gauss-Markov 
Model. 

The  GAIM-GM  model  is  a  time  dependent  physics-based  model  that  uses  the 
Ionospheric  Forecast  Model  (IFM)  as  a  background  model  upon  which  perturbations 
are  imposed  via  a  Kalman  filter  data  assimilation  scheme.  Various  data  types  can 
be  imported,  including  in  situ  electron  densities  and  line  of  sight  TEC  values  from 
satellites.  The  GAIM  model  was  also  developed  by  Utah  State  University  and  is 
currently  used  by  the  Air  Force  Weather  Agency  for  ionospheric  modeling.  GAIM 
determines  the  best  selection  of  data  sources  (for  a  given  combination  of  overall 
data  sources;  i.e.,  GPC  TEC  values)  and  applies  known  biases  and  weights  to  the 
assimilated  data.  Assimilated  data  deemed  too  different  from  the  IFM  is  discarded 
as  bad  data.  Then,  the  remaining  values  are  superimposed  onto  the  IFM  background 
using  a  sequential  least  squares  fit.  GAIM  requires  numerous  inputs  due  to  the  fact 
it  is  an  assimilative  model;  it  can  assimilate  bottom-side  electron  density  profiles 
from  ionosondes,  slant  and  vertical  TEC  from  GPS  stations,  DMSP  in-situ  electron 
densities,  and  line  of  sight  UV  emissions  measured  by  DMSP  ( Scherliess  et  al,  2006). 
The  overall  GAIM  workflow  is  shown  in  Figure  3. 

Like  IFM,  GAIM  output  is  also  in  the  form  of  a  3-D  grid.  However,  GAIM  only 
assimilates  and  produces  values  related  to  electron  densities.  Outputs  include  NmE, 
hmE,  NnnF2,  hmU2,  electron  densities,  and  slant  and  vertical  TEC  values.  GAIM  has 
a  resolution  of  4.67°  latitude  and  15°  longitude,  the  coarsest  resolution  of  any  of  the 
models  considered  in  this  effort.  A  regional  mode  is  available  with  a  resolution  of 
1°  latitude  and  3.75°  longitude,  but  only  after  GAIM  has  been  run  globally.  Lastly, 
GAIM  can  be  run  in  a  forecast  mode  up  to  24  hours  out.  GAIM  is  available  online 
at  http:/ /ccmc.gsfc. nasa.gov/models/modelinfo. php?model=USLI-GAIM. 
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Figure  3.  GAIM-GM  model  run  flowchart  depicting  how  data  is  assimilated  into  the 
model. 


2.3  Electron  Density  and  Electron  Temperature  and  the  ISS 

The  ISS  is  an  ideal  candidate  for  testing  the  validity  of  the  ionospheric  models 
because  of  its  short  orbital  period  (91  minutes),  orbital  inclination  (51.7°),  and  loca¬ 
tion  in  the  ionosphere.  Its  short  orbital  period  allows  for  many  orbits  worth  of  data 
to  be  analyzed  for  a  given  day  across  different  seasons  and  locations.  The  ISS  also 
provides  a  unique  in-situ  data  source  for  electron  density  and  electron  temperature 
data  just  above  the  F2  peak. 

The  ISS  faces  unique  manned  spacecraft  charging  hazards  due  to  the  high  voltage 
of  its  solar  arrays  ( Ferguson  et  al,  2003).  It  has  been  determined  that  a  potential 
difference  of  40V  or  greater  represents  a  hazard  to  astronauts  during  EVAs,  and  that 
the  largest  potential  differences  occur  when  the  ISS  is  traveling  through  a  region  of 
cold,  high-density  plasma. 

To  mitigate  this  hazard,  two  plasma  contactor  units  are  powered  on  for  the  du¬ 
ration  of  all  EVAs  and  replenishes  the  electrons  collected  by  the  high-voltage  solar 
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arrays,  reducing  the  potential  differences  to  safe  levels.  While  the  plasma  contactor 
units  completely  mitigate  these  charging  hazards,  a  secondary  hazard  mitigation  plan 
must  be  in  place  in  case  one  of  the  plasma  contactor  units  fails.  In  order  to  monitor 
the  charging  environment  on  the  ISS,  NASA  installed  the  Floating  Potential  Mea¬ 
surement  Unit  (FPMU)  on  the  ISS  in  2006.  The  FPMU  was  created  by  Utah  State 
University’s  Space  Dynamics  Laboratory  and  was  installed  on  the  ISS  in  2006. 

2.4  Floating  Potential  Measurement  Unit 

The  floating  potential  measurement  unit  (FPMU),  shown  in  Figure  5,  was  de¬ 
signed  to  be  capable  of  providing  redundant  measurements  of  various  plasma  parame¬ 
ters  and  consists  of  four  probes:  a  Floating  Potential  Probe  (FPP),  a  Wide-Sweeping 
Langmuir  Probe  (WLP),  a  Narrow-Sweeping  Langmuir  Probe  (NLP),  and  a  Plasma 
Impedance  Probe  (PIP).  The  FPP  measures  the  ISS  floating  potential  (the  potential 
difference  between  the  ISS  and  the  surrounding  plasma),  while  the  WLP  and  NLP 
measure  the  ISS  floating  potential,  the  electron  density,  and  the  electron  temperature. 
The  values  measured  by  the  WLP  and  NLP  have  been  shown  to  agree  within  10% 
( Barjatya  et  al,  2009). 

Electron  densities  and  temperatures  are  computed  by  measuring  the  current  re¬ 
sulting  from  a  range  of  applied  voltages,  and  then  applying  a  complicated  fitting 
process  to  the  resulting  data  to  derive  electron  densities  and  temperatures  ( Wright 
et  al,  2008).  This  is  sometimes  known  as  a  “graphical  method”  and  each  region  of  the 
resulting  current-voltage  (or  I-V)  plot  is  treated  separately.  The  three  main  regions 
in  the  I-V  curves  are  the  ion  saturation  region  (where  the  net  ion  current  dominates), 
electron  retarding  region  (where  the  ion  current  is  negligible,  but  there  is  enough  of 
a  negative  potential  to  partially  repel  electrons),  and  the  electron  saturation  region 
(where  the  net  electron  current  dominates).  An  example  of  a  typical  I-V  curve  from 
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Figure  4.  A  sample  I-V  curve  created  by  a  current-voltage  sweep  performed  by  the 
FPMU  WLP.  Note  the  three  distinct  regions  in  the  plot.  The  voltage  at  which  the 
current  changes  polarity  (9.2  V)  is  the  potential  of  the  ISS.  This  plot  follows  the 
“Langmuir  Convention,”  with  ion  current  recorded  as  negative  and  electron  current  as 
positive  (Adapted  from  Wright  et  al.  2008). 


this  analysis  is  shown  in  Figure  4. 

There  are  many  steps  involved  in  deriving  electron  densities  and  temperatures 
from  the  raw  FPMU  data,  which  will  be  summarized  here.  A  more  thorough  treat¬ 
ment  is  provided  in  Wright  et  al.  (2008).  First,  the  applied  voltage  that  causes  a 
change  in  current  polarity  is  found  and  assumed  to  be  the  measured  potential  of  the 
ISS  (i.e. ,  the  floating  potential).  This  change  in  polarity  defines  the  boundary  be¬ 
tween  the  ion  saturation  region  and  the  electron  retarding  region  of  the  I-V  curve. 
Then,  a  correction  is  applied  to  the  current  values  to  account  for  the  photoelectron 
current.  A  least  squares  fit  is  then  applied  to  the  ion  saturation  region  (assuming 
a  linear  relationship  between  ion  current  and  the  applied  voltage)  to  determine  the 
ion  current  baseline,  which  can  then  be  subtracted  from  the  measured  current  to 
determine  the  electron  current.  Finally,  the  voltage  of  the  surrounding  plasma  is 
determined  by  finding  the  voltage  corresponding  to  the  maximum  value  of  the  first 
derivative  of  the  electron  current. 

The  calculated  ion  current  is  then  assumed  to  be  equivalent  to  the  ion  ram  current 
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(Iram)  i  since  the  ISS  orbits  at  a  speed  greater  than  the  ion  thermal  speed  (but  below 
the  electron  thermal  speed).  Since  the  ISS  is  moving  faster  than  the  ion  thermal  speed, 
it  is  assumed  that  ions  collected  by  the  ISS  are  not  dependent  on  the  temperature  of 
the  ions.  By  making  this  assumption,  the  ion  density  can  be  calculated  by  using  the 
relation 


Iram  —  q  ■  Ni  •  VlSS  '  ALP  (1) 

where  q  is  the  electronic  charge,  Nt  is  the  ion  density,  Vjss  is  the  magnitude  of 
the  ISS  velocity,  and  Alp  is  the  cross  sectional  area  of  the  Langmuir  probe.  For 
this  study,  the  spherical  WLP  was  used  for  calculating  ion  densities,  so  the  cross 
section  was  constant.  The  electron  density  values  provided  by  NASA  for  this  effort 
were  actually  derived  ion  densities.  Comparisons  of  WLP  and  NLP  electron  and 
ion  densities  to  COSMIC  satellite  in-situ  electron  densities  have  shown  the  WLP 
ion  densities  to  correlate  much  better  to  the  COSMIC  electron  densities  than  the 
electron  densities  from  either  probe,  with  the  electron  densities  generally  significantly 
less  than  the  derived  ion  densities  ( Gurgew ,  2011).  Since  the  ionosphere  is  a  quasi¬ 
neutral  plasma,  the  ion  and  electron  densities  should  theoretically  be  the  same.  NASA 
believes  this  discrepancy  is  due  to  the  differences  in  deriving  electron  and  ion  densities 
from  the  raw  data  ( Gurgew ,  2011).  Due  to  these  issues,  NASA  has  decided  to  report 
the  derived  ion  densities  as  electron  densities  ( Minow ,  2012). 

Electron  temperatures  are  then  calculated  using  the  derived  electron  current  and 
potential  of  the  surrounding  plasma  (plasma  space  potential).  It  is  assumed  that 
the  electrons  in  the  electron  retarding  region  are  Maxwellian  and  follow  a  Boltzmann 
relation  for  electron  current  (Ie)  given  by 


Ie 


Ieo  ■  exp 


q(V  -  VsP) 
kTe 


(2) 
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where  q  is  the  electron  charge,  k  is  Boltzmann’s  constant,  Vsp  is  the  plasma  space 
potential,  Te  is  the  electron  temperature,  and  Ieo  is  the  random  electron  thermal 
current  and  is  given  by 


where  Ne  is  the  electron  density,  Ap  is  the  area  of  the  Langmuir  probe,  and  me  is 
the  electron  mass.  These  relations  are  then  used  in  the  curve  fitting  process  in  the 
electron  retardation  region  to  determine  the  electron  temperature  values.  Average  fit 
errors  are  calculated  only  for  electron  temperatures,  and  are  generally  less  than  1%, 
while  variations  in  the  electron  density  and  temperature  data  suggest  errors  closer 
to  10%  for  both,  although  a  detailed  error  analysis  has  yet  to  be  performed  ( Minow , 
2012).  While  not  used  in  this  effort,  electron  densities  are  derived  from  the  I-V 
curve  data  by  substituting  the  derived  electron  temperature  value  and  potential  of 
the  surrounding  plasma  into  Equation  3  and  solving  for  electron  temperature. 

While  electron  temperatures  and  densities  measured  by  the  WLP  and  NLP  have 
been  shown  the  be  within  10%  of  one  another,  it  is  difficult  to  validate  the  values 
themselves  due  to  the  lack  of  in-situ  electron  density  data  at  ISS  altitudes.  A  limited 
validation  effort  was  performed  by  Coffey  et  al.  (2008)  by  comparing  FPMU  measured 
data  to  electron  densities  measured  by  the  TIMED-SEE  satellite,  ionosondes,  and  an 
incoherent  scattered  radar  (ISR)  site  within  a  certain  temporal  and  spatial  range 
relative  to  the  ISS.  Then,  IRI-2007  was  used  to  determine  the  average  expected  ratio 
between  the  FPMLI  values  and  the  other  values  (all  at  different  altitudes  than  the 
ISS),  and  relative  differences  were  calculated  to  to  see  how  often  the  percent  difference 
between  the  two  sets  of  values  were  within  the  expected  ratios.  The  TIMED-SEE 
values  were  within  the  accepted  difference  value  28%  of  the  time,  while  ionosonde 
data  differences  were  within  the  expected  value  74%  of  the  time  and  ISR  differences 
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FPP 


Figure  5.  FPMU  Diagram  (Adapted  from  Gurgew  2011) 

were  within  the  expected  value  66%  of  the  time.  Electron  temperature  values  were 
compared  in  the  same  manner  for  the  ISR  site,  and  the  differences  were  within  the 
expected  value  26%  of  the  time.  It  is  believed  that  the  conjunctive  comparison  process 
played  a  significant  role  in  the  larger  observed  differences  than  expected. 

To  provide  a  backup  charging  hazard  mitigation  plan  for  an  EVA,  NASA  collects 
FPMU  electron  densities  for  2-3  days  prior  to  a  scheduled  EVA.  Then,  it  also  produces 
IRI-2007  model  output  for  the  same  period  using  ISS  ephemeris  data  and  determines 
how  well  the  FPMU  data  is  correlating  to  IRI  output.  Then,  NASA  determines  if 
the  FPMU  data  correlates  more  strongly  with  unchanged  IRI  output,  or  if  an  offset 
applied  to  the  IRI  output  provides  a  better  fit.  Then,  the  “corrected”  (if  necessary) 
IRI  data  is  input  into  a  computer  model  called  the  Plasma  Interaction  Model  (PIM) 
which  simulates  ISS  charging  conditions  based  on  various  ISS  electrical  configurations. 
The  PIM  output  indicates  whether  any  steps  need  to  be  taken  to  alter  the  electrical 
current  path  of  any  of  the  solar  panels  to  reduce  the  charging  hazard,  or  if  the  standard 
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charging  configuration  is  safe  based  on  the  ionospheric  conditions  from  the  IRI-2007 
output  ( Minow  et  ai,  2002). 

During  initial  efforts  to  determine  the  charging  effects  and  levels  the  ISS  would 
experience,  it  was  discovered  that  more  electrons  would  be  collected  by  the  ISS  solar 
arrays  for  lower  electron  temperature  values  than  for  higher  ones.  This  is  contrary 
to  the  assumed  relation  where  a  conductor  with  a  positive  bias  should  collect  an 
electron  current  proportional  to  the  electron  density  and  to  the  square  root  of  the 
electron  temperature  (i.e.,  the  thermal  current  density).  Due  to  the  fact  that  only 
the  edges  of  the  solar  cells  are  exposed  to  the  plasma  and  are  partially  “hidden”  from 
the  plasma  due  to  construction  and  spacing  of  the  cells,  a  potential  barrier  is  created 
that  “preferentially  excludes  high-temperature  electrons  and  admits  low-temperature 
electrons”  ( Ferguson ,  2009).  By  analyzing  electron  temperature  and  density  data 
near  peak  charging  times  (generally  eclipse  exit)  measured  by  a  device  called  the 
Floating  Potential  Probe  (FPP),  which  was  the  predecessor  to  the  FPMU  on  the  ISS 
and  performed  a  similar  function,  the  Ferguson- Morton  relation  was  established  and 
is  given  by 


4>  =  -2.6907  TVg'1057  exp(— 8.02  Te)  (4) 

where  <h  is  the  charging  potential  (V),  Ne  is  the  electron  density  (^/m3),  and  Te 
is  the  electron  temperature  (eV).  While  this  relation  was  originally  calculated  when 
the  ISS  only  had  two  solar  arrays  (it  currently  has  four),  the  same  general  relation 
is  expected  to  hold,  while  the  specific  coefficients  themselves  may  be  different  in  the 
current  configuration  (Minow,  2012).  From  Equation  4  we  see  that  small  changes  in 
electron  temperatures  can  cause  large  charging  potentials.  Due  to  the  general  inverse 
relationship  between  electron  density  and  temperature  near  the  F2  peak,  some  of  the 
lowest  electron  temperature  values  are  associated  with  high  electron  densities,  com- 
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pounding  the  charging  effect.  While  charging  dne  to  magnetic  induction  provides  a 
significant  contribution  to  the  ISS  charging  levels,  it  is  not  dependent  on  the  densities 
and  temperatures  of  the  surrounding  plasma.  The  largest  observed  potential  differ¬ 
ences  tend  to  occur  during  eclipse  exit  (the  times  for  which  Equation  4  are  assumed 
to  be  valid)  and  are  dependent  on  the  electron  temperatures  and  densities,  clue  to  the 
fact  that  the  solar  arrays  operate  at  a  positive  voltage  which  acts  to  attract  electrons 
and  drive  the  ISS  potential  more  negative  ( Wright  et  al. ,  2008). 

2.5  Previous  Efforts 

A  limited  set  of  FPMU  ion  densities  were  compared  to  the  IRI  and  GAIM  models 
by  ( Barjatya  et  al,  2009)  in  2009,  and  while  they  did  not  provide  quantitative  data 
regarding  the  differences  between  the  data,  both  models  were  in  general  agreement 
with  the  in-situ  measurements,  with  GAIM  tracking  the  FPMU  values  better.  Their 
results  show  that  in  situ  density  measurements  agree  better  with  the  GAIM  model 
than  the  IRI  model  in  general,  which  is  expected  clue  to  GAIMs  data  assimilative 
nature.  Temperatures  were  only  compared  to  the  IRI  model,  but  also  were  said  to  be 
similar  to  the  in  situ  values. 
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III.  Methodology 


3.1  Representative  Ionospheric  Conditions 

In  order  to  thoroughly  test  the  validity  of  the  models,  a  wide  range  of  ionospheric 
conditions  were  considered  for  this  effort.  Solar  activity  and  geomagnetic  storms 
have  a  significant  impact  on  ionospheric  conditions,  so  representative  solar  and  geo¬ 
magnetic  activity  conditions  were  used  to  compare  the  performance  of  these  models. 
Geomagnetic  storming  conditions  are  given  by  the  Kp  index,  which  is  a  global  index 
that  measures  the  amount  of  variation  in  the  horizontal  magnetic  field  measured  at 
earth  and  varies  from  0-9.  Solar  activity  levels  are  given  by  the  F10.7  index,  which 
measures  the  daily  observed  flux  from  the  sun  at  a  wavelength  of  10.7  cm  in  solar 
flux  units  ( sfu  =  10”22  W  m~2  s ).  Three  time  periods  were  evaluated  during  low 
solar  activity  (solar  minimum):  geomagnetic  quiet  conditions,  moderate  geomagnetic 
storming,  and  strong  geomagnetic  storming.  Similarly,  the  same  three  geomagnetic 
conditions  were  evaluated  during  periods  of  moderate/high  solar  activity  (solar  max¬ 
imum),  for  a  total  of  six  cases.  Only  time  periods  with  at  least  five  consecutive  days 
of  FPMU  data  were  chosen  for  this  study  in  order  to  provide  an  adequate  sample 
size  for  each  period  and  to  examine  the  performance  of  the  models  during  onset  or 
recovery  periods.  Table  1  shows  all  of  the  dates  selected,  along  with  the  maximum 
Kp  and  F10.7  value  for  the  period. 

3.2  Floating  Potential  Measurement  Unit  Data 

All  data  from  the  FPMU  is  transmitted  through  the  Ku  frequency  band,  which  is 
the  high-bandwidth  (video)  communications  band  for  the  ISS.  However,  data  or  video 
from  other  ISS  activities  often  take  precedent  over  FPMU  data  transmission,  resulting 
in  gaps  in  FPMU  data.  Additionally,  all  of  the  Ku-band  data  is  relayed  through  the 
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Table  1.  Periods  of  study  chosen  for  this  effort  and  their  associated  geomagnetic  and 
solar  activity  levels. 


Run 

Day  of  Year 

Dates 

Year 

Geomagnetic 
Activity  (Kp) 

Solar 

Activity  (F10.7) 

1 

253-256 

10-13  Sept 

2007 

Quiet  (1) 

Quiet  (67) 

2 

70-74 

10-14  March 

2008 

Moderate  (4) 

Quiet  (68) 

3 

95-99 

5-9  April 

2010 

Storming  (7-) 

Quiet  (78) 

4 

60-64 

1-5  March 

2011 

Moderate  (5+) 

Active  (130) 

5 

120-125 

30  Apr  -  5  May  2011 

Moderate  (5-) 

Moderate  (110) 

6 

292-296 

19-23  October 

2011 

Quiet  (2+) 

Active  (164) 

Tracking  and  Data  Relay  Satellite  System  (TDRSS)  to  stations  on  the  ground  in  the 
US;  due  to  TDRSS  communications  limitations  over  Asia,  no  Ku-band  data  can  be 
transmitted  when  the  ISS  is  at  40  —  90°  longitude  ( Minow ,  2012).  However,  due  to  the 
short  orbital  period  of  the  ISS,  it  is  still  possible  to  obtain  data  from  a  representative 
range  of  latitudes  at  various  local  times.  An  example  of  ISS  orbital  data  coverage  is 
provided  in  Figure  6. 

NASA  provided  the  FPMU  data  in  text  hie  format.  Each  text  hie  output  included 
the  year,  decimal  day  of  year  (DOY),  latitude  (geographic),  longitude,  altitude,  elec¬ 
tron  density  (#/m3),  electron  temperature,  and  orientation  of  the  ISS  (pitch  in  de¬ 
grees)  on  each  line.  Missing  electron  density  or  electron  temperature  values  were 
identified  with  a  value  of  -1.  NASA  used  the  WLP  for  electron  density  values,  while 
they  used  the  NLP  for  the  electron  temperature  values;  these  were  chosen  because 
they  had  been  determined  to  be  the  most  accurate  FPMU  sensors  for  each  respective 
parameter. 


3.3  Running  the  Models 


Each  of  the  ionospheric  models  compared  in  this  study  is  operated  very  differently. 
Unsurprisingly,  the  climatological  models  (IRI)  required  the  least  amount  of  effort 
and  information  to  run,  while  the  physics-based  models  required  slightly  more  work 
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Figure  6.  FPMU  data  coverage  for  10-12  September  2007.  Note  the  zone  of  exclusion 
(lack  of  data  points)  over  40  —  90°  longitude. 

to  run,  and  the  data  assimilation  model  required  the  most  effort  to  run  due  to  the 
need  to  download  and  prepare  data  for  assimilation  into  the  model.  The  processes 
required  to  operate  each  of  the  models  are  described  below. 

3.3.1  IRI  1990  and  IRI  2012. 

IRI-90  and  IRI- 12  were  both  run  using  FORTRAN  codes  (one  for  each  of  the 
two  models)  to  ingest  ISS  times,  locations,  and  heights  into  IRI-90  and  IRI-12  to 
determine  each  models  predicted  electron  density,  electron  temperature,  and  total 
electron  content  at  each  point  in  space  and  time  where  FPMU  data  existed.  Total 
electron  content  values  were  calculated  by  integrating  IRI  predicted  electron  density 
values  at  each  FPMU  latitude  and  longitude  from  the  surface  of  the  earth  to  1000 
km  in  10  km  steps.  NrnFz  and  hrriF‘2  values  were  determined  by  finding  the  peak 
electron  density  value  at  each  of  those  10  km  steps  and  the  height  at  which  that  peak 


22 


value  was  found.  Input  flags  used  for  both  models  are  provided  in  Appendix  A.  Since 
these  models  do  not  run  in  a  three-dimensional  mode  like  the  rest  of  the  models,  the 
output  values  were  valid  for  the  input  ISS  location  and  no  interpolation  of  model 
results  was  required. 

The  output  text  hies  included  year,  decimal  DOY  (UT),  latitude,  longitude, 
altitude,  FPMU-measured  electron  density  (#/m3),  IRI- predicted  electron  density 
(^/m3),  FPMU-measured  electron  temperature  (Kelvin),  IRI-predicted  electron  tem¬ 
perature  (Kelvin),  local  time  (decimal  hours),  ISS  orientation  (degrees),  and  IRI- 
predicted  TEC,  NmF2 ,  and  hmF2  values  on  each  line. 

3.3.2  CTIPe. 

All  of  the  CTIPe  runs  completed  for  this  effort  were  completed  by  using  NASA’s 
Community  Coordinated  Modeling  Center  (CCMC).  Due  to  the  unavailability  of 
quality-controlled  and  hourly-averaged  ACE  data  for  some  of  the  run  periods  for 
input  into  CTIPe,  the  CCMC  ran  all  six  runs  using  real-time  ACE  data  for  consistency 
amongst  all  runs.  This  also  represents  more  of  a  realistic  set  of  operational  conditions 
for  using  CTIPe  as  a  mission  planning  tool  for  EVAs,  since  ACE  averaged  data  would 
not  be  available  in  time  to  be  used  at  or  near  real-time. 

NASA  CCMC  provided  the  CTIPe  data  in  .tar  hies  for  each  day,  each  of  which 
contained  three  output  hies  for  the  day  (in  NetCDF  hie  format),  along  with  input 
hies.  Time  steps,  height  grid  values,  latitude  grid  values,  and  longitude  grid  values 
from  the  output  were  also  used.  Since  CTIPe  produces  one  of  each  type  of  output 
hie  for  each  day,  some  of  the  model  output  is  in  a  four-dimensional  grid  (i.e.,  a  three- 
dimensional  output  grid  for  each  fifteen  minute  output  time  step)  within  the  data  hie. 
All  output  hies  were  downloaded  from  the  CCMC  servers.  NmF2  and  hmF2  values 
were  determined  by  finding  the  peak  electron  density  values  in  the  electron  density 
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grids  at  each  timestep  and  the  height  at  which  those  peak  values  were  found;  this 
resulted  in  2-D  NrnF-2  and  hmF2  grids  with  the  same  resolution  as  the  electron  density 
grids. 


3.3.3  IFM. 

All  of  the  IFM  runs  for  this  effort  were  performed  remotely  on  a  USU  computer 
with  IFM  and  GAIM  installed.  To  complete  an  IFM  run,  desired  model  output  dates 
were  modified  in  a  text  hie,  which  was  then  read  by  the  model.  All  model  drivers 
were  automatically  downloaded  and  input  into  IFM. 

Since  the  IFM  model  output  acts  as  the  baseline  for  the  GAIM  model  upon  which 
perturbations  are  applied  using  assimilated  data,  additional  “warm-up”  days  were 
run  outside  the  time  period  for  each  run  as  part  of  the  process  of  running  the  GAIM 
models.  IFM  outputs  hies  in  the  NetCDF  format,  similar  to  CTIPe;  however,  IFM 
produces  a  different  output  hie  at  each  fifteen  minute  time  step.  IFM-predicted 
NmF2  and  hmF2  values  were  calculated  in  the  same  manner  described  in  3.3.2. 

3.3.4  GAIM. 

Like  IFM,  all  GAIM  runs  were  performed  remotely  on  a  USU  computer  with 
GAIM  installed.  Prior  to  executing  a  GAIM  model  run,  a  number  of  steps  had  to  be 
completed  to  download  and  prepare  data  for  assimilation  into  the  model.  The  GAIM 
model  had  to  be  run  for  one  day  prior  to  the  desired  output  time  period,  and  IFM 
output  had  to  exist  for  a  day  prior  to  that.  So,  if  hve  days  of  model  output  were 
desired,  GAIM  had  to  be  run  for  a  total  of  six  days,  and  IFM  had  to  be  run  for  a 
total  of  seven  days. 

In  order  to  better  understand  the  impact  different  data  sources  have  on  GAIM 
performance,  GAIM  was  run  in  two  different  modes:  with  two  data  sources  (GPS  TEC 
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and  ionosonde  data),  and  with  three  data  sources  (GPS  TEC,  ionosonde  data,  and 
DMSP  in-situ  electron  densities).  The  runs  using  two  data  sources  are  identified  in 
this  effort  as  GAIM  2,  while  the  runs  using  three  data  sources  are  identified  as  GAIM 
3.  Additionally,  GAIM  3  was  run  in  forecast  mode  to  create  three,  six,  twelve,  and 
twenty-four  hour  forecasts.  GAIM  3  was  chosen  because  it  was  assumed  to  perform 
the  best  due  to  the  additional  data  assimilation  source,  particularly  an  in-situ  electron 
density  source.  These  forecast  runs  were  included  to  potentially  provide  more  options 
to  NASA  regarding  EVA  mission  planning  by  examining  how  well  GAIM  can  predict 
values  in  the  future. 

The  first  step  in  running  GAIM  is  to  prepare  the  input  data  for  assimilation. 
This  is  done  by  executing  a  number  of  command  line  scripts  created  by  USU  that 
download  GPS  TEC,  ionosonde,  or  DMSP  data  after  modifying  text  files  to  reflect 
the  days  for  which  data  is  required.  GPS  TEC  data  is  downloaded  from  NOAA, 
NASA,  and  UCSD  data  sources,  and  DMSP  electron  density  data  and  ionosonde 
data  are  downloaded  from  NOAA.  Next,  the  downloaded  input  data  is  reduced,  or 
processed,  into  a  format  that  GAIM  can  use  through  command  line  scripts.  Further 
information  about  this  process  can  be  found  in  ( Scherliess  et  al,  2006).  Additionally, 
these  scripts  determine  the  optimal  number  of  those  sources  geographically  to  provide 
the  best  global  coverage  and  write  those  sources  to  a  text  file.  Those  text  files  are 
then  read  by  the  GAIM  program  to  determine  which  data  to  assimilate. 

Configuration  files  must  also  be  edited  to  specify  which  data  sources  will  be  as¬ 
similated,  the  time  steps  of  the  output,  and  whether  the  model  will  be  run  in  forecast 
mode  (and  if  so,  what  the  forecast  output  time  step  will  be).  Then,  the  main  GAIM 
scripts  can  be  run.  Like  IFM,  all  GAIM  output  files  are  in  NetCDF  format  with  the 
date  and  time  the  output  file  is  specified  for  in  the  filename.  For  this  effort,  fore¬ 
cast  output  files  were  identified  by  a  suffix  added  to  the  filename.  GAIM-predicted 
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NrnF-2  and  hmF2  values  were  calculated  in  the  same  manner  described  in  3.3.2. 

3.4  Importing  and  Interpolating  Data 

All  data  was  imported  into  Matrix  Lab  software  (MATLAB  version  7.13.0.564) 
for  analysis.  Scripts  were  written  for  this  effort  to  import,  process,  and  interpolate 
(when  necessary)  FPMLI  and  model  output.  These  scripts  are  described  below. 

3.4.1  Binomial  Filter. 

First,  FPMLI  and  IRI  datasets  were  imported  into  MatLAB.  Then,  all  of  the  miss¬ 
ing  electron  density  and  electron  temperature  data  were  removed,  with  the  remaining 
“good”  data  for  each  parameter  saved  with  its  corresponding  time  and  location  values 
(i.e.,  sets  of  separate  times/locations  for  electron  density  and  electron  temperature). 

A  binomial  filter  was  then  used  to  filter  out  noise  in  the  FPMU  data.  This  was 
done  using  a  five-point  filter  with  weights  determined  as  outlined  in  ( Anbury  and  Luk , 
1995)  and  shown  in  Equation  1,  where  FPMUi  represents  a  FPMLI  data  point  and 
FPMU *i  represents  the  filtered  values.  The  binomial  filter  approximates  a  Gaussian 
and  provides  a  better  low-pass  Liter  than  a  simple  moving  average.  Then,  a  Liter  was 
applied  to  remove  data  points  where  all  Lve  data  points  used  in  the  binomial  Liter 
were  not  separated  by  one  second  each;  this  was  done  to  ensure  the  Liter  was  not 
applied  across  temporal  “gaps”  in  FPMU  data.  This  led  to  a  slight  reduction  in  the 
number  of  valid  data  points  used.  A  Lve-weight  Liter  was  chosen  to  be  large  enough 
to  smooth  out  Lne  noise  and  reduce  the  amount  of  points  discarded  from  the  analysis 
set.  The  resulting  subset  of  valid  FPMU  time  and  location  data  was  saved  as  in  the 
previous  manner  and  used  for  the  rest  of  the  processes  described  below. 
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FPMU  *i 


FPMUi-2  +  4  (FPMUi-i)  +  6(FPMUi)  +  4  {FPMUi+1)  +  FPMUi+2 

16 

(5) 


3.4.2  Interpolation  and  Data  Processing. 

Since  CTIPe,  IFM,  and  GAIM  all  produce  three  dimensional  output,  model- 
predicted  electron  densities  and  temperatures  needed  to  be  interpolated  to  the  FPMU 
times  and  locations.  Additionally,  CTIPe,  IFM  and  GAIM  all  produced  output  in 
15  minute  intervals  for  this  effort  for  a  total  of  96  three-dimensional  grids  for  a  given 
day,  while  the  FPMU  outputs  data  at  a  cadence  of  roughly  1  Hz,  when  available.  All 
interpolation  and  data  processing  was  done  in  MatLAB. 

CTIPe  has  a  latitude  limitation  in  its  electron  temperature  output,  with  values  of 
0  K  at  ±  50°  geomagnetic  latitude.  These  values  were  excluded  from  all  calculations. 

Temporally,  the  filtered  FPMU  data  was  broken  up  into  15-minute  periods  of 
time  (matching  the  output  of  the  3-D  models)  and  compared  to  the  previous  model 
output  (in  time).  For  instance,  FPMU  data  from  both  1205  and  1210  UT  would  be 
compared  to  the  3-D  model  outputs  from  1200  UT.  This  represents  a  more  realistic 
scenario  for  the  real-time  use  of  the  models  operationally  by  NASA,  with  the  most 
current  real-time  output  used.  Further,  in  most  cases  the  ionosphere  does  not  change 
significantly  at  a  given  point  within  a  15-minute  period,  with  the  exception  of  local 
dawn  and  dusk,  while  the  ISS  may  encounter  significant  ionospheric  environmental 
changes  in  that  same  period  due  to  its  high  orbital  speed. 

Once  the  FPMU  data  was  separated  into  15-minute  segments,  FPMU  latitudes, 
longitudes  and  heights  within  each  15-minute  period  were  input  into  a  3-D  linear 
interpolation  scheme  in  Matlab,  using  Matlab’s  built-in  “interp3”  function.  This  pro¬ 
duced  an  interpolated  electron  density,  electron  temperature,  or  Total  Electron  Con- 
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tent  value  at  each  FPMU  location  using  the  3-D  (or  2-D,  in  the  case  of  TEC,  hmF-2, 
and  NmF-2 )  output  from  the  closest  previous  15-minute  time  step  as  described  above. 
Height  values  were  not  required  for  the  interpolation  of  TEC,  hrriF-z ,  or  TVmi^  values 
because  they  do  not  vary  with  height.  This  interpolation  scheme  created  interpolated 
model  output  for  each  FPMU  electron  temperature  and  density  value,  as  well  as  TEC, 
NmF2,  and  hmF2  values  for  all  of  the  times  and  locations  where  FPMU  data  existed. 

3.5  Analyzing  Model  Performance 

After  all  of  the  FPMU  and  model  data  was  imported  and  interpolated,  it  was 
output  to  text  hies  for  further  analysis  by  other  Matlab  utilities  created  for  this 
effort.  Additionally,  the  open  source  plotting  program  gnuplot  (version  4.6)  was  used 
to  create  all  of  the  plots  in  this  effort. 

3.5.1  Electron  Densities,  hmF2,NmF2,  and  Total  Electron  Content. 

Electron  densities  were  compared  using  mean  absolute  percent  difference  (MAPD), 
mean  percent  difference  (MPD),  and  Pearson’s  correlation  coefficient  values  as  de¬ 
scribed  below.  Since  TEC,  hmF2 ,  and  NmF2  were  not  measured  by  the  FPMU, 
analysis  involving  those  parameters  was  limited  to  calculating  and  comparing  aver¬ 
age  values.  This  helped  identify  overall  trends  and  whether  differences  in  electron 
densities  between  models  were  related  to  the  models  predicting  an  overall  difference 
in  the  number  of  electrons  or  simply  a  different  height  distribution  of  the  electrons. 

3.5.2  Electron  Temperatures. 

Electron  temperatures  were  compared  in  two  different  ways  for  this  effort.  First, 
all  electron  temperature  models  were  compared  to  FPMU  data  using  the  methods 
previously  described  where  model  data  exists.  This  resulted  in  some  gaps  in  CTIPe 


electron  temperature  values  at  high  latitudes.  Next,  all  the  models  were  compared  for 
latitudes  ±  38°  latitude,  which  is  the  largest  latitude  range  that  CTIPe  has  electron 
temperature  data  for.  While  this  likely  doesn’t  represent  an  operational  scenario,  it 
provides  a  way  to  directly  compare  model  performance  across  the  same  times  and 
locations. 

3.6  Statistical  Analysis 

There  are  a  myriad  of  different  statistical  methods  that  can  be  used  to  analyze 
the  performance  of  a  model  relative  to  measured  data.  For  this  effort,  mean  ab¬ 
solute  percent  difference  (MAPD),  mean  percent  difference  (MPD),  and  Pearsons 
product-moment  correlation  coefficient  (r)  were  calculated  between  FPMU  data  and 
the  models  for  each  day.  MAPE  was  used  to  determine  the  average  percent  error  for  a 
given  day  and  shows  which  model  produces  output  that  most  closely  matches  FPMU 
data,  while  the  MPE  values  identify  overall  biases  (i.e.,  positive  or  negative)  present 
in  the  models.  While  root-mean-square  errors  values  are  commonly  reported  in  sim¬ 
ilar  validation  efforts,  they  are  omitted  here  because  they  do  not  provide  meaningful 
information  about  the  scale  of  the  errors  in  relation  to  the  FPMU  data,  which  is  the 
focus  for  ISS  mission  planning. 

Lastly,  Pearsons  product-moment  correlation  coefficient  provides  a  measure  of  how 
well  each  of  the  models  correlates  (linearly)  to  the  FPMU  data.  This  is  similar  to  the 
method  used  by  NASA  for  EVA  mission  planning  as  described  in  Chapter  2. 

Further,  each  of  these  metrics  was  broken  up  by  latitude  region  and  local  time  of 
day  to  more  closely  examine  model  performance.  Performance  was  evaluated  by  day 
for  each  run,  and  for  each  run  as  a  whole.  Additionally,  to  evaluate  performance  in  a 
more  general  sense,  statistics  were  calculated  for  the  entire  set  of  data. 
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3.6.1  MAPD  and  MPD. 


In  order  to  gain  a  quantitative  understanding  of  the  differences  in  FPMU  and 
model  values,  percent  difference  values  were  calculated.  Mean  absolute  percent  dif¬ 
ference  (MAPD)  was  used  to  determine  the  mean  error  across  the  data  subset  eval¬ 
uated  (Equation  6).  Smaller  MAPD  values  are  better,  and  indicate  that  the  model 
predicted  values  are  not  significantly  different  than  the  FPMU  measured  values. 

Mean  percent  difference  (MPD)  was  also  calculated  to  evaluate  overall  biases 
(positive  or  negative)  in  the  models  (Equation  7).  By  definition,  MPD  values  are  less 
than  or  equal  to  MAPD  values  in  magnitude.  It  is  important  to  look  at  both  MPD 
and  MAPD  values  because  a  small  MPD  value  (in  magnitude)  does  not  necessarily 
correspond  to  a  small  MAPD  value.  A  small  MPD  value  (in  magnitude)  simply  means 
that  the  model  in  question  does  not  have  a  tendency  to  show  a  large  bias.  In  general, 
a  small  bias  is  preferred.  However,  a  MPD  (bias)  that  is  close  in  magnitude  to  the 
MAPD  (average  error)  may  be  more  easily  correctable  in  the  models,  since  that  would 
indicate  a  nearly  constant  positive  or  negative  bias  and  may  be  able  to  be  somewhat 
hxed  by  multiplying  model  values  by  a  correction  factor. 
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3.6.2  Correlation  Coefficient. 

In  addition  to  MAPD  and  MPD,  Pearson’s  Correlation  Coefficient  is  useful  be¬ 
cause  it  evaluates  how  well  model  data  correlates  linearly  to  FPMU  data.  This  is 
important  because  a  consistent  bias  in  one  of  the  models  could  perhaps  be  over¬ 
come  by  a  scaling  factor  if  the  model  correlated  strongly  enough.  Strong  correlation 
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indicates  the  models  are  capturing  the  overall  trends  in  the  ionosphere. 

This  coefficient  is  given  by  the  equation  ( Navidi ,  2006): 

n  _  _ 

X]  ( FPMUi  -  FPMU )  (Modek  -  Model ) 

=  (8) 

(Models  —  Model)2 

Correlation  coefficient  values  can  vary  from  -1  to  1.  The  higher  the  magnitude,  the 
stronger  the  linear  correlation.  Zero  values  indicate  the  two  data  sets  are  completely 
uncorrelated.  Sometimes  the  correlation  coefficient  value  of  a  large  set  of  data  can  be 
stronger  than  the  correlation  values  of  subsets  of  the  same  set  of  data;  this  can  happen 
when  outlier  data  points  in  the  data  set  are  poorly  correlated  and  skew  the  subset  of 
data.  When  included  in  a  larger  set  of  data,  their  overall  affect  on  the  correlation  is 
diminished,  which  is  why  larger  data  sets  tend  to  be  better  for  evaluating  correlation. 


rn  /  n 

a/E  (FPMUi  -  FPMU)  Jy: 
V  i= 1  V  i=  1 


31 


IV.  Results 


4.1  Chapter  Overview 

The  results  of  the  analysis  methodology  described  in  Chapter  3  are  presented  here. 
First,  the  performance  of  each  of  the  models  is  described  based  on  each  run.  Then, 
overall  trends  in  the  data  are  presented.  Due  to  the  large  amount  of  information  ana¬ 
lyzed  in  this  effort,  only  representative  plots  are  provided  here.  Electron  density  and 
electron  temperature  plots  and  data  not  presented  here  are  available  upon  request. 
Values  highlighted  in  green  represent  the  best  performance  values  for  a  given  period, 
while  red  values  indicate  the  worst  performance  values.  For  correlation  coefficient  val¬ 
ues,  the  strongest  values  are  highlighted  in  green,  while  the  weakest  or  most  inversely 
correlated  values  are  highlighted  in  red.  For  MAPD,  the  smallest  values  are  high¬ 
lighted  in  green,  while  the  largest  values  are  highlighted  in  red.  Finally,  the  smallest 
absolute  MPD  values  (i.e.,  the  models  with  the  smallest  biases)  are  highlighted  in 
green,  while  the  largest  magnitude  MPD  values  are  highlighted  in  red. 

Electron  temperature  results  are  shown  based  on  the  overall  data  available  to 
each  model  unless  otherwise  noted;  due  to  limitations  in  CTIPe  electron  temperature 
coverage,  the  analysis  for  CTIPe  used  a  smaller  subset  of  data  points  than  the  other 
three  models.  However,  for  completeness  all  of  the  electron  temperature  models  were 
evaluated  using  the  least  restrictive  set  of  latitudes.  The  results  of  the  analysis  using 
a  common  subset  of  latitudes  referred  to  as  “CTIPe-restricted  data”  in  this  effort  and 
available  upon  request.  Differences  between  the  two  sets  of  metrics  are  mentioned  if 
they  are  significant. 
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4.2  Overall  Trends 


In  this  section,  trends  in  the  overall  electron  temperature  and  electron  density 
data  sets  are  examined.  This  is  done  to  provide  an  overall  representation  of  model 
performance  across  many  different  geomagnetic  and  solar  conditions.  Then,  in  the 
sections  that  follow,  model  performance  during  specific  geomagnetic  and  solar  activity 
conditions  will  be  examined  and  compared  to  the  model  performance  as  a  whole. 
Average  hmF2 ,  NmF2,  and  TEC  values  across  all  days  are  presented  in  Table  2.  All 
of  the  hmF2  values  are  within  12  km  of  each  other  and  are  close  to  our  nominal 
expected  value  of  300  km.  IRI-12  predicted  the  lowest  hmF2 ,  while  IFM  predicted 
the  highest  hmF2.  Additionally,  the  use  of  the  DMSP  data  source  in  GAIM  3  did 
not  significantly  shift  the  F2  peak  compared  to  the  GAIM  2  hmF2  values,  increasing 
it’s  height  by  0.61  km  on  average.  CTIPe  predicted  the  lowest  average  NmF2  and 
TEC  values  of  all  of  the  models,  almost  half  that  of  IFM,  which  predicted  the  largest 
values.  As  a  result,  we  would  expect  CTIPe  to  predict  the  lowest  electron  densities 
and  IFM  to  predict  the  highest  electron  densities  of  all  of  the  models. 

Table  2.  Overall  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

290.27 

5.06 

15.52 

IRI-12 

285.61 

4.50 

13.15 

CTIPe 

295.51 

2.90 

8.09 

IFM 

296.43 

6.24 

15.76 

GAIM  2 

293.35 

5.70 

15.31 

GAIM  3 

293.96 

5.64 

15.34 

GAIM  3  3-hr 

295.59 

5.74 

15.40 

GAIM  3  6-hr 

296.81 

5.88 

15.55 

GAIM  3  12-hr 

297.51 

5.94 

15.62 

GAIM  3  24-hr 

297.51 

5.94 

15.63 
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4.2.1  Electron  Density. 


Overall,  GAIM  2  performed  the  best  across  all  of  the  available  electron  density 
data  with  the  strongest  correlation  (0.870)  and  the  third  lowest  MAPD  (57%),  al¬ 
though  GAIM  3  performed  essentially  just  as  well  (Table  3).  This  result  was  expected 
because  of  GAIM’s  assimilative  nature.  All  of  the  GAIM  3  forecasts  showed  strong 
correlation  to  FPMU  values  (greater  than  0.828),  although  they  tended  to  have  the 
second  largest  MAPD  values  behind  IFM,  likely  due  to  the  fact  they  were  forecasts. 
All  the  GAIM  3  forecasts  had  a  positive  bias  in  relation  to  the  real-time  GAIM  mod¬ 
els.  The  two  physics-based  models  (IFM  and  CTIPe)  performed  the  worst  overall, 
although  they  still  showed  generally  strong  correlation  as  a  whole  (0.740  and  0.701, 
respectively).  IFM  had  the  largest  MAPD  by  far  at  95%,  likely  driven  by  its  strong 
positive  bias  of  64%,  which  was  somewhat  expected  from  the  ATni^and  TEC  results. 

All  of  the  models  except  CTIPe  tended  to  overestimate  electron  densities,  which 
underestimated  electron  densities  by  28%  on  average,  partially  due  to  the  fact  that 
it  does  not  seem  to  model  the  Appleton  Anomaly.  IRI-12  showed  very  little  positive 
bias  overall  (7%),  while  all  of  the  other  models  tended  to  overestimate  by  at  least 
30%.  While  the  GAIM  3  12  hour  forecast  tended  to  perform  slightly  worse  than  the 
GAIM  3  24  hour  forecast,  the  differences  were  negligible. 

Looking  at  diurnal  trends,  GAIM  2  and  GAIM  3  performed  the  best  by  far  during 
the  daytime  (Table  4).  All  of  the  GAIM  models  (including  the  forecasts)  showed 
stronger  correlation  during  the  daytime  than  the  other  models,  with  strong  correlation 
values  of  at  least  0.841.  CTIPe  performed  the  worst  during  the  daytime,  with  the 
lowest  correlation  (0.709)  and  the  third  highest  MAPD  (52%).  CTIPe  also  showed  a 
strong  negative  bias  (MPD)  during  the  daytime  (-48%);  CTIPe’s  poor  performance 
during  the  daytime  and  its  tendency  to  under  predict  electron  densities  were  partially 
due  to  the  fact  that  CTIPe  does  not  seem  to  model  the  Appleton  Anomaly,  which 
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Table  3.  Total  performance  of  ionospheric  model  predicted  electron  density  values  in 
relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.840 

64 

38 

IRI-12 

0.843 

46 

7 

CTIPe 

0.701 

62 

-28 

IFM 

0.740 

95 

64 

GAIM  2 

0.870 

57 

32 

GAIM  3 

0.867 

56 

30 

GAIM  3 

3-hr 

0.839 

66 

41 

GAIM  3 

6-hr 

0.830 

71 

48 

GAIM  3 

12-hr 

0.828 

75 

52 

GAIM  3 

24-hr 

0.830 

74 

52 

produces  a  “double  peak”  in  electron  density,  as  shown  in  Figure  7.  GAIM  2  and  3 
had  a  very  weak  negative  bias  (less  than  1%),  while  the  rest  of  the  models  exhibited 
a  positive  bias  during  the  daytime.  Figures  8  and  9  show  examples  of  good  and  bad 
model  performance  across  many  ISS  orbits,  respectively. 

Table  4.  Diurnal  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


(r) 

Day 

(%) 

(%) 

Night 
(r)  (%) 

(%) 

IRI-90 

0.836 

62 

35 

0.761 

67 

40 

IRI-12 

0.841 

42 

7 

0.771 

49 

7 

CTIPe 

0.709 

52 

-48 

0.484 

72 

-10 

IFM 

0.767 

69 

35 

0.596 

119 

91 

GAIM  2 

0.881 

33 

-1 

0.775 

80 

63 

GAIM  3 

0.878 

33 

0 

0.773 

77 

59 

GAIM  3  3-hr 

0.851 

44 

13 

0.722 

86 

67 

GAIM  3  6-hr 

0.843 

50 

20 

0.714 

92 

74 

GAIM  3  12-hr 

0.841 

52 

23 

0.713 

96 

79 

GAIM  3  24-hr 

0.843 

51 

22 

0.716 

96 

80 

At  nighttime,  GAIM  2  and  GAIM  3  still  had  the  best  correlation,  but  its  MAPD 
rose  to  at  least  77%  and  it  tended  to  over  predict  electron  density  values  by  at 
least  59%.  IRI-12  performed  the  best  at  night,  with  the  third  strongest  correlation 
(0.771)  and  the  smallest  MAPD  (42%).  IFM  performed  the  worst  at  night,  with 
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UT  (hh) 

Figure  7.  Model  Predicted  Electron  Density  Values  versus  FPMU  Values  for  7  April 
2010  (during  geomagnetic  storming  and  low  solar  activity).  Model  values  are  in  blue, 
FPMU  values  are  in  red.  In  bottom  graph,  local  time  is  plotted  in  orange  and  latitude 
is  plotted  in  black. 
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Figure  8.  Model  Predicted  Electron  Density  Values  versus  FPMU  Values  for  5  May 
2011  (during  moderate  geomagnetic  and  solar  activity)  showing  good  model  perfor¬ 
mance  overall.  Model  values  are  in  blue,  FPMU  values  are  in  red.  In  bottom  graph, 
local  time  is  plotted  in  orange  and  latitude  is  plotted  in  black. 
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Figure  9.  Model  Predicted  Electron  Density  Values  versus  FPMU  Values  for  Day 
64  2011  (during  moderate  geomagnetic  and  high  solar  activity)  showing  poor  model 
performance  overall.  Model  values  are  in  blue,  FPMU  values  are  in  red.  In  bottom 
graph,  local  time  is  plotted  in  orange  and  latitude  is  plotted  in  black. 
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Local  Time  (H) 


the  second  weakest  correlation  (0.596)  and  the  largest  MAPD  by  far  (119%).  As 
expected,  GAIM’s  overall  performance  trends  followed  IFM’s,  due  to  the  fact  IFM  is 
used  as  a  background  for  IFM.  All  of  the  models  except  CTIPe  tended  to  overestimate 
electron  density  at  night,  particularly  IFM  (91%).  While  still  exhibiting  a  negative 
bias,  CTIPe’s  bias  was  much  smaller  in  magnitude  (10%)  at  night,  when  the  Appleton 
Anomaly  does  not  occur.  Additionally,  there  was  a  significant  increase  in  overall  bias 
(greater  than  +38%)  in  all  of  the  models  except  the  two  IRI  models,  likely  due  to  their 
empirical,  climatological  nature.  The  weakened  correlation  and  increase  in  positive 
bias  at  night  is  in  part  due  to  plasma  bubbles  and  dawn  density  depictions  created  at 
night  by  equatorial  spread  F  conditions.  Currently,  none  of  the  models  considered  in 
this  effort  account  for  these  effects  due  to  the  tenuous  and  volatile  nature  of  plasma 
bubbles  as  well  as  the  spatial  resolution  of  the  three  dimensional  models. 

Regionally,  IRI-12  performed  the  best  at  low-latitudes,  although  GAIM  2  and 
GAIM  3  had  slightly  stronger  correlation  values  (but  much  larger  MAPD  values  as 
well),  as  shown  in  Table  5.  Again  all  of  the  models  except  CTIPe  showed  a  positive 
bias,  with  IFM  and  GAIM  exhibiting  the  strongest  positive  bias.  CTIPe  and  IFM 
performed  the  worst  overall  at  low  latitudes,  with  the  two  weakest  correlation  values 
and  the  two  highest  MAPD  values. 

At  mid-latitudes,  GAIM  3  performed  the  best,  showing  the  strongest  correlation 
(0.870)  and  the  second  smallest  MAPD  (46%).  CTIPe  and  the  GAIM  models  showed 
an  improvement  in  performance  at  mid-latitudes,  while  IFM  and  the  IRI  models 
exhibited  a  weaker  correlation  (but  smaller  MAPD)  at  mid-latitudes.  This  decrease 
in  MAPD  was  likely  due  to  the  lack  of  plasma  bubbles  and  the  Appleton  Anomaly  in 
the  mid-latitude  region,  which  the  models  often  struggled  to  accurately  predict.  IFM 
performed  the  worst  overall  at  mid-latitudes,  with  the  weakest  correlation  (0.643)  and 
the  largest  MAPD  (90%).  All  of  the  models  except  CTIPe  again  showed  a  positive 
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Table  5.  Regional  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.805 

71 

36 

0.768 

59 

39 

IRI-12 

0.807 

54 

5 

0.787 

39 

8 

CTIPe 

0.617 

78 

-7 

0.727 

51 

-44 

IFM 

0.690 

102 

55 

0.643 

90 

70 

GAIM  2 

0.831 

72 

45 

0.870 

47 

22 

GAIM  3 

0.827 

70 

42 

0.870 

46 

21 

GAIM  3  3-hr 

0.795 

78 

47 

0.809 

57 

36 

GAIM  3  6-hr 

0.787 

82 

50 

0.791 

64 

46 

GAIM  3  12-hr 

0.784 

84 

52 

0.787 

68 

52 

GAIM  3  24-hr 

0.785 

84 

53 

0.793 

67 
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bias  at  mid-latitudes,  with  a  slight  increase  in  positive  bias  for  IFM  and  the  IRI 
models  at  mid-latitudes  (compared  to  low-latitudes). 

4.2.2  Electron  Temperature. 

IRI-12  performed  the  best  of  all  of  the  models  at  predicting  electron  temperatures, 
with  a  moderate  correlation  of  0.697  and  MAPD  of  22%;  IRI-90  performed  essentially 
the  same  with  a  slightly  lower  correlation  and  slightly  higher  MAPD  (Table  6).  IFM 
performed  the  worst  of  all  of  the  models,  which  was  somewhat  expected  given  the  age 
of  its  electron  temperature  model  ( Schunk ,  2012).  It  had  a  low  correlation  of  0.328 
(but  a  slightly  stronger  0.510  using  the  CTIPe-restricted  data  set)  and  a  MAPD 
of  42%.  Other  than  the  aforementioned  difference  in  IFM  correlation,  there  was 
a  negligible  difference  between  using  the  full  set  of  data  and  the  CTIPe-restricted 
set  of  ephemeris  data.  CTIPe  was  the  only  model  to  exhibit  a  positive  bias,  albeit 
a  very  weak  one  (1%).  These  model  bias  values  display  the  previously  mentioned 
inverse  relationship  between  electron  density  and  electron  temperature;  the  model 
that  predicted  the  lowest  electron  temperature  values  (IFM)  is  also  the  model  that 
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predicted  the  highest  electron  density  values  on  average.  Conversely,  the  model  that 
tended  to  under  predict  electron  densities  the  most  (CTIPe)  was  the  only  one  with  a 
positive  bias.  Figures  10  and  11  show  examples  of  good  and  bad  model  performance 
across  many  ISS  orbits,  respectively. 

Table  6.  Total  performance  of  ionospheric  model  predicted  electron  temperature  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.700 

24 

-19 

IRI-12 

0.697 

22 

-19 

CTIPe 

0.567 

24 

1 

IFM 

0.328 

42 

-40 

All  of  the  models  performed  worse  during  the  daytime,  due  to  the  fact  that  electron 

temperatures  fluctuate  the  most  during  the  daytime,  when  electron  densities  also 

fluctuate  the  most.  The  best  model  correlation  coefficient  value  (IRI-90)  was  a  weak 

0.357  (Table  7).  IRI-90  performed  the  best  overall  during  the  day,  also  producing  the 

lowest  MAPD  (17%).  IFM  again  performed  the  worst,  with  very  weak  correlation 

(0.064)  and  the  highest  MAPD  (42%).  IFM’s  poor  daytime  performance  was  likely 

related  to  its  tendency  to  over  predict  electron  density  values  then,  leading  to  larger 

electron  cooling  rates.  CTIPe  exhibited  a  negligible  negative  bias,  while  the  other 

models  tended  to  under  predict  electron  temperatures  by  at  least  10%. 

Table  7.  Diurnal  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.357 

17 

-10 

0.499 

31 

-28 

IRI-12 

0.335 

19 

-16 

0.532 

26 

-22 

CTIPe 

0.179 

21 

-1 

0.327 

26 

3 

IFM 

0.064 

48 

-47 

0.233 

36 

-32 

All  of  the  models  showed  better  correlation  at  night,  when  IRI-12  performed  the 
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Figure  10.  Model  Predicted  Electron  Temperature  Values  versus  FPMU  Values  for  7 
April  2010  (during  geomagnetic  storming  and  low  solar  activity)  showing  good  model 
performance  overall.  Model  values  are  in  blue,  FPMU  values  are  in  red.  In  bottom 
graph,  local  time  is  plotted  in  orange  and  latitude  is  plotted  in  black. 
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Figure  11.  Model  Predicted  Electron  Temperature  Values  versus  FPMU  Values  for 
1  March  2011  (during  moderate  geomagnetic  and  high  solar  activity)  showing  poor 
model  performance  overall.  Model  values  are  in  blue,  FPMU  values  are  in  red.  In 
bottom  graph,  local  time  is  plotted  in  orange  and  latitude  is  plotted  in  black. 
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Local  Time  (H) 


best  with  the  strongest  correlation  (0.532)  and  and  lowest  MAPD  (27%).  IFM  again 
performed  the  worst  overall  at  night.  The  IRI  models  had  a  larger  negative  bias  at 
night,  while  the  magnitude  of  IFM’s  negative  bias  decreased  at  night.  This  decrease 
in  negative  bias  at  night  (when  IFM  tended  to  over  predict  electron  density  values 
more  than  during  the  daytime)  was  likely  due  to  the  direct  relationship  between  tem¬ 
perature  and  scale  height.  At  night,  electron  densities  are  greatly  diminished,  so  the 
large  cooling  rates  from  collisions  with  relatively  cold  ions  that  dominate  during  the 
day  are  no  longer  as  important.  As  a  result,  scale  height  plays  a  larger  role  in  deter¬ 
mining  electron  densities  (since  electron  production  essentially  goes  away  at  night), 
and  as  scale  height  increases,  densities  decrease  more  slowly  with  altitude,  leading 
to  larger  densities  just  above  the  F2  peak.  However,  the  night  time  plasma  bubbles 
seen  in  the  electron  density  data  had  a  large  effect  on  the  electron  density  MAPD 
and  MPD  values  during  the  night  time  and  could  have  skewed  the  apparent  effect  of 
scale  height,  since  plasma  bubble  effects  were  not  seen  in  the  electron  temperature 
data. 

Regionally,  1RI-12  performed  the  best  of  the  models  at  low  latitudes,  attaining  the 
strongest  correlation  (0.710)  and  the  second  lowest  MAPD  value  (27%),  as  seen  in 
Table  8.  CTIPe  had  the  smallest  MAPD  (25%),  but  had  the  weakest  correlation  of  the 
models  at  low  latitudes  (0.502).  IFM  again  had  the  largest  MAPD  (34%).  Similarly, 
all  of  the  models  showed  a  negative  bias,  while  CTIPe  had  a  slight  positive  bias 
(6%).  All  of  the  models  except  CTIPe  showed  stronger  correlation  at  low  latitudes, 
even  though  all  of  the  models  except  IFM  had  larger  MAPD  values  there  than  at 
mid-latitudes. 

At  mid-latitudes,  IFM  performed  the  worst  overall,  showing  weak  correlation 
(0.215)  and  a  MAPD  nearly  twice  as  large  as  the  other  models  (45%).  Correlation 
improved  in  IFM  to  0.429  when  using  the  CTIPe-restricted  set  of  ISS  ephemeris  data, 
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Table  8.  Regional  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Low  Latitude  (  A 

<  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.676 

32 

-30 

0.623 

21 

-14 

IRI-12 

0.710 

27 

-26 

0.606 

20 

-16 

CTIPe 

0.502 

25 

6 

0.529 

23 

-1 

IFM 

0.592 

34 

-33 

0.215 

45 

-42 

as  well  as  a  negligible  2%  decrease  in  MAPD.  IFM’s  change  in  performance  at  mid¬ 
latitudes  (worsening  with  increasing  latitude)  is  likely  due  to  its  mid/high-latitude 
electron  temperature  model  and  the  “blending”  region  that  overlaps  with  much  of  the 
mid-latitude  region  considered  here.  Again  the  overall  inverse  relationship  between 
electron  density  and  electron  temperature  is  seen  in  1FM,  which  tended  to  over  pre¬ 
dict  electron  densities  more  at  mid-latitudes  and  under  predict  electron  temperatures 
more  there.  1R1-90  performed  the  best  at  mid-latitudes  with  the  strongest  correlation 
(0.630)  and  a  MAPD  value  of  20%,  although  lRI-12  performed  essentially  the  same. 
The  general  latitude  trends  seen  in  the  regional  electron  density  performance  held 
here,  with  CTIPe’s  performance  improving  at  mid  latitudes  (with  the  caveat  that  it 
couldn’t  predict  electron  temperature  values  across  all  of  the  mid  latitudes  considered 
by  the  other  models),  and  all  of  the  other  models  exhibiting  worse  performance  at 
mid-latitudes  than  at  low  latitudes.  CTIPe  showed  the  most  constant  performance 
across  both  regions,  but  the  two  IRI  models  still  performed  better  in  both. 

4.3  Run  1  Results  -  Low  Geomagnetic  and  Solar  Activity 

Run  1  represented  solar  and  geomagnetic  quiet  conditions  and  occurred  during  10- 
12  September  2007.  The  F10.7  index  stayed  relatively  constant  at  67  for  the  duration 
of  the  period,  while  the  Kp  index  remained  at  or  below  1  (Figure  12).  All  of  the  models 
seemed  to  struggle  the  most  with  this  combination  of  geomagnetic  and  solar  activity, 
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likely  due  to  the  deep  solar  minimum  experienced  during  this  period.  The  deep  solar 
minimum  of  2007-2008  was  a  very  unique  period.  As  a  result  of  the  extremely  low 
solar  EUV  flux,  the  lowest  thermospheric  densities  ever  recorded  occurred  during 
this  period  ( Emmert  et  al,  2010).  Additionally,  there  was  generally  poor  correlation 
between  EUV  irradiance  and  the  F10.7  index  ( Chen  et  al,  2011),  which  all  of  the 
models  use  as  a  proxy  for  solar  activity.  Analysis  by  Bilitza  et  al.  (2012)  suggests 
that  there  was  a  fundamental  change  in  ionospheric  composition  during  this  period, 
especially  in  the  topside  ionosphere  profile,  due  to  the  decreased  densities  and  EUV 
flux. 

Additionally,  this  period  of  solar  minimum  was  unique  because  a  previously  undis¬ 
covered  plasma  depletion  phenomenon  was  observed.  This  phenomenon,  described 
by  Huang  et  al.  (2009),  is  the  presence  of  broad  plasma  depletions  (much  larger  in 
horizontal  extent  than  plasma  bubbles)  at  night  near  the  equator.  Since  they  are 
so  broad,  they  are  not  formed  in  the  same  way  as  plasma  bubbles,  but  it  is  not  yet 
fully  understood  what  causes  them.  However,  it  is  believed  that  they  are  a  result 
of  cooling  in  the  ionosphere  and  thermosphere,  especially  in  the  equatorial  region 
(Huang  et  al.,  2009).  Based  on  analyzing  electron  density  plots  during  this  period, 
the  FPMU  encountered  these  depletions  during  most  of  its  orbits  during  this  run  and 
during  many  of  the  orbits  during  Run  2. 

Average  hmF2,  NrnF2,  and  TEC  values  for  the  period  are  presented  in  Table 
9,  and  show  some  distinct  differences  in  relation  to  the  overall  averages.  Here  we 
see  the  predicted  NmF2  and  TEC  values  are  much  lower  than  the  overall  averages, 
with  NmF2  and  TEC  values  nearly  reduced  by  half  as  a  result  of  the  low  F10.7 
and  Kp  index  values.  Here  IFM  predicted  the  largest  NmF2 ,  while  IRI-12  predicted 
the  smallest.  For  TEC,  CTIPe  and  IRI-90  predicted  the  smallest  and  largest  TEC 
values,  respectively,  suggesting  significant  differences  in  the  relative  electron  density 
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Figure  12.  Run  1  Kp  and  F10.7  Index  Values  for  10-13  September  2007  ( sfu  = 

10~22fF  TO-2  s). 


distributions  between  the  models.  Lastly,  the  predicted  values  are  much  lower 

(approximately  20  km  lower  than  the  overall  average),  due  to  the  general  decrease  in 
electron  densities  predicted  during  this  period. 


4.3.1  Electron  Density. 

Electron  density  MAPD,  MPD  and  correlation  coefficients  are  presented  in  Table 
10.  All  of  the  models  overestimated  electron  densities  during  this  period  by  greater 
than  64%.  IFM  had  the  worst  performance,  with  a  MAPD  value  of  170%  for  the 
period.  IFM  also  had  the  second  weakest  correlation  for  the  period  (0.522),  while 
CTIPe  had  the  weakest  correlation  (0.391)  and  third  highest  MAPD  values. 

As  a  whole,  1R1-12  and  GAIM  3  performed  the  best  during  this  period  of  solar  and 
geomagnetic  quiet  conditions,  with  GAIM  3  posting  the  second  strongest  correlation 
overall  (0.686)  and  the  second  lowest  lowest  MAPD  (84.00%).  The  use  of  an  additional 
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Table  9.  Run  1  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

274.54 

3.18 

10.01 

IRI-12 

270.90 

2.71 

7.82 

CTIPe 

250.81 

2.87 

6.20 

IFM 

272.32 

4.37 

8.80 

GAIM  2 

266.35 

3.05 

6.82 

GAIM  3 

267.64 

2.90 

6.92 

GAIM  3  3-hr 

268.99 

3.13 

7.31 

GAIM  3  6-hr 

270.66 

3.43 

7.73 

GAIM  3  12-hr 

272.17 

3.55 

7.82 

GAIM  3  24-hr 

272.24 

3.55 

7.84 

GAIM  data  source  in  GAIM  3  brought  a  slight  reduction  in  MAPD  (6.3%)  and  a 
negligible  difference  in  correlation  when  compared  to  GAIM  2.  IRI-12  had  the  lowest 
MAPD  value  (64%)  but  had  a  weaker  correlation  value  (0.620). 

Unsurprisingly,  all  of  the  GAIM  forecasts  performed  better  in  both  MAPD  and 
correlation  than  IFM  (since  IFM  is  used  as  the  “background”  for  the  assimilated 
data),  and  tended  to  perform  worse  as  the  forecast  increased.  This  trend  can  be  seen 
in  general  throughout  all  of  the  runs,  with  the  exception  that  the  24-hour  GAIM 
3  forecast  had  a  slightly  smaller  MAPD  value  (0.94%  difference)  than  the  12-hour 
forecast. 

Table  10.  Run  1  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.593 

117 

113 

IRI-12 

0.615 

65 

56 

CTIPe 

0.384 

107 

80 

IFM 

0.508 

171 

155 

GAIM  2 

0.684 

87 

78 

GAIM  3 

0.683 

81 

71 

GAIM  3  3-hr 

0.607 

99 

90 

GAIM  3  6-hr 

0.580 

117 

108 

GAIM  3  12-hr 

0.575 

125 

117 

GAIM  3  24-hr 

0.579 

124 

117 
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Looking  at  diurnal  trends  (Table  11),  all  of  the  models  performed  significantly 
worse  during  local  nighttime  conditions,  with  IFM  and  CTIPe  posting  MAPD  values 
of  202%  and  154%,  respectively.  All  of  the  models  exhibited  a  strong  positive  bias 
at  night  as  well.  The  two  physics-based  models  also  had  weak  negative  correlation 
coefficient  values  (-0.090  for  CTIPe  and  -0.041  for  IFM),  indicating  a  weak  inverse 
correlation  and  very  bad  correlation  to  FPMU  electron  density  values  at  night  during 
solar  and  geomagnetic  quiet  conditions.  IRI-12  performed  the  best  of  the  models 
during  the  nighttime,  although  it  still  showed  weak  correlation  (0.421)  and  had  a 
high  MAPD  value  (65%).  In  fact,  all  of  the  models  except  IRI-12  had  MAPD  values 
greater  than  110%  during  the  nighttime  and  correlation  coefficient  values  weaker  than 
0.325.  A  significant  reason  the  models  didn’t  perform  well  at  night  was  due  to  broad 
plasma  depletions,  which  the  FPMU  measured  on  most  of  the  orbits  during  this  run. 
An  example  of  a  broad  plasma  depletion  is  provided  in  Figure  13  at  approximately 
0400  UT.  None  of  the  models  currently  account  for  these  depictions,  as  evidenced  in 
the  plot. 

Table  11.  Diurnal  Run  1  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.574 

111 

108 

0.264 

121 

115 

IRI-12 

0.592 

64 

60 

0.421 

65 

53 

CTIPe 

0.459 

31 

-7 

-0.090 

154 

135 

IFM 

0.496 

122 

116 

-0.041 

202 

179 

GAIM  2 

0.696 

33 

19 

0.325 

121 

116 

GAIM  3 

0.693 

33 

19 

0.308 

111 

103 

GAIM  3  3-hr 

0.598 

56 

46 

0.113 

127 

117 

GAIM  3  6-hr 

0.571 

74 

67 

0.058 

143 

134 

GAIM  3  12-hr 

0.569 

79 

73 

0.030 

153 

144 

GAIM  3  24-hr 

0.571 

77 

71 

0.034 

154 

146 

GAIM  2  and  GAIM  3  performed  the  best  during  the  daytime  (Table  11),  with  the 
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Figure  13.  Model  Predicted  Electron  Density  Values  versus  FPMU  Values  for  12 
September  2007  (during  low  geomagnetic  and  solar  activity).  Model  values  are  in  blue, 
FPMU  values  are  in  red.  In  bottom  graph,  local  time  is  plotted  in  orange  and  latitude 
is  plotted  in  black.  Note  the  broad  plasma  depletion  that  occurs  around  0400  UT  that 
none  of  the  models  account  for. 
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second  and  third  lowest  MAPD  values  (33%)  and  the  strongest  correlation  (0.696  and 
0.693,  respectively).  IFM  performed  the  worst  during  the  daytime,  with  an  MAPD 
value  of  122%  and  a  correlation  of  0.496.  CTIPe  had  both  the  best  MAPD  value 
(31%)  and  the  weakest  correlation  value  (0.459),  indicating  that  even  though  it  had 
the  lowest  errors,  the  other  models  were  tracking  the  trends  better.  To  better  illustrate 
the  drastic  diurnal  difference  in  performance  (and  the  effect  of  plasma  depictions), 
CTlPe’s  weak  correlation  value  during  the  day  was  still  better  than  the  strongest 
correlation  during  the  nighttime  (0.421,  by  lRI-12).  The  poor  correlation  shown 
by  all  of  the  models  during  the  daytime  is  likely  due  to  both  discontinuous  high 
FPMU  electron  density  values  near  the  Appleton  Anomaly  and  the  models  inability 
to  accurately  predict  solar  production  rates  during  such  a  low  solar  minimum. 

The  effects  of  plasma  depletions  on  the  model  performance  were  clearly  evident 
in  the  regional  analysis  as  well.  All  of  the  models  also  had  much  higher  MAPD 
values  at  low  latitudes  where  plasma  depletions  occur  (Tabic  12),  with  all  models 
averaging  at  least  a  95%  absolute  difference  value.  IFM  and  CTIPe  had  MAPD  values 
greater  than  228%.  GAIM  3  performed  the  best  overall  at  low  latitudes,  showing 
the  strongest  correlation  (0.702)  and  the  second  lowest  MAPD  value  (128%),  while 
CTIPe  performed  the  worst,  with  the  weakest  correlation  (0.390)  and  the  second 
highest  MAPD  value  (228%). 

GAIM  3  had  the  strongest  correlation  at  low  latitudes  (0.702),  but  its  correla¬ 
tion  weakened  significantly  at  mid  latitudes  (0.470),  even  though  its  MAPD  values 
decreased  by  more  than  a  factor  of  2  there.  IFM  and  all  of  the  other  GAIM  models 
showed  a  similar  trend.  Conversely,  IRI-90,  IRI-12  and  CTIPe  all  showed  stronger 
correlation  at  mid-latitudes  than  at  low-latitudes,  with  IRI-12  showing  the  strongest 
correlation  (0.766)  and  second  lowest  MAPD  value  (46%)  at  mid-latitudes.  IFM 
did  the  worst  at  mid-latitudes,  with  the  weakest  correlation  (0.430)  and  the  highest 
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Table  12.  Regional  Run  1  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Low  Latitude  (  A  <  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.592 

172 

167 

0.731 

84 

80 

IRI-12 

0.610 

96 

85 

0.766 

46 

38 

CTIPe 

0.390 

228 

216 

0.518 

33 

-3 

IFM 

0.504 

248 

215 

0.430 

125 

118 

GAIM  2 

0.702 

138 

130 

0.471 

56 

47 

GAIM  3 

0.700 

128 

119 

0.512 

52 

41 

GAIM  3  3-hr 

0.611 

151 

138 

0.478 

68 

61 

GAIM  3  6-hr 

0.581 

172 

158 

0.473 

83 

78 

GAIM  3  12-hr 

0.576 

182 

167 

0.472 

90 

86 

GAIM  3  24-hr 

0.579 

184 

170 

0.485 

88 

84 

MAPD  value  (125%).  CTIPe  had  a  slight  negative  bias  of  -3%  at  mid  latitudes,  while 
the  rest  of  the  models  exhibited  a  positive  bias.  This  negative  bias  at  mid-latitudes  is 
likely  due  to  the  relative  lack  of  plasma  depletions  at  mid-latitudes,  especially  since 
CTIPe  showed  the  largest  positive  bias  (216%)  at  low  latitudes.  GAIM  3  (and  all  of 
the  GAIM  3  forecasts)  showed  stronger  correlation  at  mid-latitudes  than  GAIM  2, 
perhaps  due  to  DMSP  data  coverage. 


4.3.2  Electron  Temperature. 

In  contrast  to  the  electron  density  performance  during  this  period,  the  models 
performed  better  during  this  period,  showing  stronger  correlation  than  they  did  as  a 
whole  across  all  runs  (Table  13).  All  models  tended  to  underestimate  FPMU  electron 
temperature  values  across  the  period  by  at  least  15%.  CTIPe  performed  the  best 
overall,  showing  the  strongest  correlation  (0.859)  by  far  and  third  lowest  MAPD  value 
(24%).  IRI-90  had  showed  stronger  correlation  and  had  a  smaller  MAPD  value  than 
IRI-12.  IFM  had  the  highest  MAPD  value  (47%)  and  the  second  weakest  correlation 
coefficient  (0.692).  IRI-12  had  the  worst  correlation  of  all  of  the  models  (0.685),  even 
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though  it  had  the  second  best  MAPD  values. 

Table  13.  Run  1  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.743 

20 

-14 

IRI-12 

0.685 

21 

-17 

CTIPe 

0.859 

24 

-23 

IFM 

0.692 

47 

-47 

Looking  at  daytime  performance,  all  of  the  models  exhibited  very  weak  correlation, 
with  all  but  1R1-90  having  a  correlation  of  less  than  0.10  (Table  14).  IFM  performed 
the  worst,  with  a  very  weak  correlation  of  0.020  and  a  MAPD  value  of  50%,  while 
IRI-90  did  the  best  with  the  strongest  (but  still  weak)  correlation  of  0.307  and  the 
lowest  MAPD  value  (19%).  All  of  the  models  had  weaker  day  time  correlation  and 
stronger  day  time  negative  biases  during  this  period,  likely  due  to  over  predicting 
electron  densities  more  than  in  the  overall  results  during  this  run. 

Table  14.  Run  1  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.307 

19 

-13 

0.752 

22 

-17 

IRI-12 

0.071 

22 

-19 

0.736 

20 

-12 

CTIPe 

0.099 

25 

-23 

0.734 

22 

-22 

IFM 

0.020 

50 

-50 

0.628 

40 

-40 

At  night,  all  of  the  models  performed  much  better;  each  model  had  a  correlation  of 
at  least  0.628.  IRI-90  again  did  the  best  overall  with  the  strongest  correlation  (0.752) 
and  the  second  lowest  MAPD  value  (22%),  while  IFM  again  did  the  worst,  with  a 
correlation  of  0.628  and  a  MAPD  of  40%.  All  of  the  models  showed  a  negative  bias 
for  both  day  and  night.  While  the  models  exhibited  very  poor  correlation  during  the 
day  time  (in  general  and  compared  to  the  night  time  values),  the  MAPD  and  MPD 
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values  improved  by  only  a  small  amount  during  night  time.  This  indicates  that  the 
models  were  not  accurately  capturing  the  trends  in  the  FPMU  electron  temperature 
data  in  the  daytime,  even  though  the  errors  were  relatively  low.  Additionally,  the 
general  improvement  in  performance  at  night  during  this  period  indicates  that  the 
larger  electron  density  errors  observed  at  night  during  this  period  were  largely  due  to 
the  higher  incidence  of  dawn  density  depletions. 

While  there  was  a  distinct  difference  between  day  and  nighttime  performance,  the 
performance  in  each  of  the  latitude  regions  was  largely  similar,  although  all  of  the 
models  except  CTIPe  tended  to  perform  better  at  low  latitudes,  consistent  with  the 
trend  seen  in  the  overall  data  (Table  15).  At  low  latitudes,  all  of  the  models  showed 
strong  correlation  values  of  at  least  0.768  (IRI-90).  CTIPe  performed  the  best,  with 
a  correlation  of  0.847  and  a  MAPD  of  21%.  IFM  had  the  second  strongest  correlation 
(0.825),  but  again  had  the  highest  MAPD  (43%).  IRI-90  performed  the  worst  at  low 
latitudes,  with  the  weakest  correlation  (0.768)  and  the  second  largest  MAPD  (32%). 

Table  15.  Run  1  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Low  Latitude  (  A 

<  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.768 

32 

-31 

0.774 

15 

-7 

IRI-12 

0.772 

25 

-24 

0.640 

20 

-14 

CTIPe 

0.847 

21 

-19 

0.892 

25 

-25 

IFM 

0.825 

43 

-43 

0.655 

48 

-48 

At  mid-latitudes,  CTIPe  showed  very  strong  correlation  (0.892),  but  had  the  third 
highest  MAPD  (25%).  IFM’s  correlation  slightly  weakened  at  mid-latitudes  to  0.665, 
and  again  had  the  highest  MAPD  (48%)  and  worst  performance  overall. 
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Figure  14.  Run  2  Kp  and  F10.7  Index  Values  for  10-14  March  2008  ( sfu  = 

10~22 W  to-2  s). 

4.4  Run  2  Results  -  Low  Solar  Activity  and  Moderate  Geomagnetic 
Storming 

Run  2  occurred  during  10-14  March  2008  and  represented  low  solar  activity  and 
moderate  geomagnetic  storming.  F10.7  values  for  the  period  stayed  constant  at  68 
for  the  entire  period,  while  Kp  index  values  fluctuated  between  1+  and  4  (Figure  14). 

Average  hmF2 ,  NmF2,  and  TEC  values  for  the  period  are  presented  in  Table 
16.  Here  we  see  slightly  larger  NmF2  and  TEC  values  than  in  Run  1  due  to  the 
increased  geomagnetic  activity,  but  they  are  still  well  below  the  overall  averages  due 
to  the  deep  solar  minimum  during  this  period.  CTIPe  again  predicted  the  lowest 
NmF2  and  TEC  values,  and  IFM  and  IRI-90  predicted  the  largest  NmF2  and  TEC 
values,  respectively.  CTIPe  also  predicted  the  lowest  hmF2 ,  while  GAIM  3  24-hr 
predicted  the  highest  value. 
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Table  16.  Run  2  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

282.15 

4.13 

13.36 

IRI-12 

275.84 

3.18 

9.78 

CTIPe 

275.53 

1.99 

4.94 

IFM 

281.70 

4.24 

9.62 

GAIM  2 

279.44 

3.73 

9.27 

GAIM  3 

280.35 

3.56 

9.33 

GAIM  3  3-hr 

282.11 

3.68 

9.33 

GAIM  3  6-hr 

283.04 

3.84 

9.47 

GAIM  3  12-hr 

283.62 

3.94 

9.54 

GAIM  3  24-hr 

283.69 

3.96 

9.73 

4.4.1  Electron  Density. 

Electron  density  MAPD,  MPD  and  correlation  coefficients  for  Run  2  are  presented 
in  Table  17.  GAIM  3  performed  the  best  of  all  the  models  in  these  conditions,  with 
the  strongest  correlation  (0.707)  and  the  second  lowest  MAPD  (44%).  All  of  the 
models  showed  worse  correlation  during  Run  2  than  across  all  data,  again  likely  due 
to  the  fact  that  the  models  performed  so  poorly  during  the  deep  solar  minimum  of 
2007-2008.  CTIPe  performed  the  worst  of  all  of  the  models,  with  a  correlation  of  0.528 
and  a  MAPD  of  52%.  All  of  the  GAIM  3  forecasts  performed  better  than  CTIPe  and 
IRI-90,  with  stronger  correlation  and  slightly  larger  MAPD  values  than  CTIPe  (but 
still  below  IRI-90).  As  seen  in  the  overall  results,  all  of  the  models  except  CTIPe 
tended  to  over  predict  electron  density  values. 

Looking  at  diurnal  trends  (Table  18),  GAIM  3  performed  the  best  during  the 
daytime,  as  in  the  overall  set  of  Run  2  data,  with  both  the  strongest  correlation 
(0.724)  and  the  lowest  MAPD  value.  CTIPe  performed  the  worst  overall  with  a 
correlation  of  0.565,  while  IRI-90  had  the  highest  MAPD  (88%).  IFM  actually  showed 
stronger  correlation  than  CTIPe  and  both  IRI  models  during  the  daytime.  All  of  the 
models  performed  worse  at  night  than  during  the  day,  again  due  to  plasma  depletions 
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Table  17.  Run  2  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.647 

72 

62 

IRI-12 

0.661 

40 

11 

CTIPe 

0.528 

52 

-44 

IFM 

0.610 

71 

45 

GAIM 

2 

0.699 

48 

21 

GAIM 

3 

0.707 

44 

17 

GAIM 

3  3-hr 

0.672 

49 

27 

GAIM 

3  6-hr 

0.654 

53 

34 

GAIM 

3  12-hr 

0.650 

57 

39 

GAIM 

3  24-hr 

0.649 

58 

40 

measured  by  the  FPMU.  The  6,  12,  and  24  hour  forecasts  of  GAIM  3  showed  slightly 

stronger  correlation  than  the  real  time  runs  of  GAIM  2  and  GAIM  3,  likely  due 

to  the  real  time  versions  of  GAIM  “trusting”  the  assimilated  data  less  than  the  IFM 

background  because  the  differences  between  the  two  values  were  deemed  too  different. 

Table  18.  Diurnal  Run  2  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.647 

88 

80 

0.570 

56 

43 

IRI-12 

0.655 

44 

26 

0.549 

37 

-5 

CTIPe 

0.565 

53 

-52 

0.528 

50 

-36 

IFM 

0.666 

56 

26 

0.519 

87 

64 

GAIM  2 

0.720 

30 

-9 

0.548 

66 

52 

GAIM  3 

0.724 

30 

-9 

0.567 

60 

44 

GAIM  3  3-hr 

0.686 

37 

6 

0.555 

62 

48 

GAIM  3  6-hr 

0.675 

42 

13 

0.575 

66 

55 

GAIM  3  12-hr 

0.674 

44 

16 

0.581 

71 

62 

GAIM  3  24-hr 

0.671 

44 

18 

0.580 

73 

64 

Regionally,  all  of  the  models  had  much  lower  MAPD  values  than  the  overall  results 
at  low  latitudes  (Table  19),  largely  due  to  fewer  plasma  depletions  measured  by  the 
FPMU  at  low  latitudes  during  this  run.  All  but  the  two  IRI  models  showed  smaller 


57 


positive  biases  at  low  latitudes  due  to  fewer  plasma  depletions  there  and  due  to 

abnormally  high  FPMU  electron  density  values  in  the  Appleton  Anomaly  ionization 

peaks.  While  both  versions  of  IRI  under  predicted  in  the  ionization  peak  areas,  they 

also  tended  to  overestimate  the  low  latitude  region  in  general  more  than  in  the  overall 

results,  similar  to  what  was  seen  during  Run  1  (also  a  period  of  solar  minimum). 

Table  19.  Regional  Run  2  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.610 

73 

59 

0.535 

72 

63 

IRI-12 

0.621 

49 

10 

0.512 

35 

11 

CTIPe 

0.449 

49 

-33 

0.392 

53 

-51 

IFM 

0.586 

61 

21 

0.220 

77 

59 

GAIM  2 

0.675 

63 

37 

0.484 

39 

11 

GAIM  3 

0.684 

57 

31 

0.517 

37 

8 

GAIM  3  3-hr 

0.640 

56 

29 

0.412 

45 

25 

GAIM  3  6-hr 

0.621 

56 

30 

0.379 

52 

36 

GAIM  3  12-hr 

0.620 

58 

32 

0.373 

56 

42 

GAIM  3  24-hr 

0.618 

60 

35 

0.375 

57 

43 

At  mid-latitudes,  all  of  the  models  showed  much  weaker  correlation  than  in  the 
overall  results,  likely  partially  due  to  the  plasma  depletions  measured  there.  GAIM 
3  performed  the  best  overall,  with  the  second  best  correlation  (0.517)  and  the  second 
lowest  MAPD  (39%).  IFM  performed  the  worst  overall  at  mid  latitudes,  with  the 
lowest  correlation  (0.220)  and  the  highest  MAPD  (77%).  CTIPe  tended  to  underesti¬ 
mate  electron  densities  the  most,  due  to  its  aforementioned  tendency  to  under  predict 
the  plasma  depletions.  IFM  over  predicted  more  at  mid  than  low  latitudes  because  it 
predicted  the  mid  latitude  plasma  depletions  the  worst  of  all  the  models  during  this 
run. 
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4.4.2  Electron  Temperature. 


The  overall  electron  temperature  performance  of  the  models  during  this  run  was 
very  similar  to  the  overall  electron  temperature  results,  with  the  exception  of  CTIPe, 
which  showed  a  significant  improvement  (Table  20).  CTIPe  had  the  best  electron 
temperature  performance,  with  a  strong  correlation  of  0.821  and  the  lowest  electron 
temperature  MAPD  value  (14%)  of  any  model  for  any  run.  IFM  again  performed  the 
worst,  with  weak  correlation  (0.351)  and  the  largest  MAPD  by  a  factor  of  two  (41%). 

Table  20.  Run  2  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.776 

20 

-9 

IRI-12 

0.762 

19 

-13 

CTIPe 

0.821 

14 

0 

IFM 

0.351 

41 

-37 

All  of  the  models  showed  slightly  worse  performance  during  the  day  than  in  the 
overall  results,  and  poor  performance  in  general  (Table  21).  The  highest  correlation 
(IRI-12)  was  a  weak  0.287,  and  IFM  was  nearly  completely  uncorrelated,  with  a 
value  of  -0.075.  However,  all  of  the  models  tended  to  perform  slightly  better  at  night 
than  for  the  overall  results  during  these  conditions,  with  IRI-12  again  performing 
the  best  with  the  strongest  correlation.  CTIPe  had  correlation  values  of  0.181  and 
0.580  during  day  and  night,  respectively,  but  had  an  overall  correlation  value  much 
higher  than  both.  This  is  likely  due  to  outlier  data  points  in  both  data  sets  that 
skewed  the  correlation  for  each  data  set,  but  had  less  of  an  impact  on  the  correlation 
of  all  of  the  data  points  from  the  run  as  a  whole.  As  the  number  of  points  in  a 
dataset  increases,  the  effect  that  outlier  data  points  has  on  the  correlation  coefficient 
diminishes  ( Navidi ,  2006). 

Regionally,  all  of  the  models  performed  generally  well  at  low  latitudes,  and  all 
produced  better  results  than  in  the  overall  low  latitude  results  (Table  22).  CTIPe  and 


59 


Table  21.  Run  2  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.212 

15 

0 

0.663 

25 

-18 

IRI-12 

0.287 

16 

-11 

0.702 

22 

-15 

CTIPe 

0.181 

15 

1 

0.580 

12 

-1 

IFM 

-0.075 

46 

-46 

0.304 

37 

-28 

IFM  actually  had  the  two  strongest  correlation  values  of  all  the  models,  with  CTIPe 
performing  the  best  overall.  All  of  the  models  performed  better  at  mid  latitudes 
except  for  IFM,  which  again  showed  weak  correlation  at  mid  latitudes  due  to  the 
electron  temperature  model  it  uses. 

Table  22.  Run  2  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Low  Latitude  (  A  <  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.721 

27 

-21 

0.739 

17 

-3 

IRI-12 

0.697 

27 

-26 

0.737 

16 

-8 

CTIPe 

0.776 

14 

0 

0.815 

14 

-1 

IFM 

0.754 

35 

-35 

0.212 

44 

-38 

4.5  Run  3  Results  -  Low  Solar  Activity  and  Geomagnetic  Storming  Con¬ 
ditions 

This  run  occurred  during  5-9  April  2010,  and  featured  very  low  solar  activity  with 
geomagnetic  storming  early  in  the  run  period  and  then  gradually  decreasing  (Figure 
15)  to  quiet  conditions. 

Average  hmF2 ,  NmF2,  and  TEC  values  for  the  period  are  presented  in  Table 
23.  Larger  average  NmF2  and  TEC  values  are  observed  during  this  period  due  to 
the  increased  geomagnetic  storming.  This  is  due  to  an  increase  in  electrons  in  the 
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Figure  15.  Run  3  Kp  and  F10.7  Index  Values  for  5-9  April  2010  ( sfu  =  10_221F  m~2  s ). 

ionosphere  coming  down  from  the  polar  caps  ( Schunk  and  Nagy ,  2009).  CTIPe  again 
predicted  NrnF2  and  TEC  values  less  than  half  that  of  the  rest  of  the  models,  while 
GAIM  2  and  GAIM  3  predicted  the  largest  NmF2  and  IRI-90  predicted  the  largest 
TEC.  There  is  also  a  wider  range  of  hmF2  values  during  this  run  than  in  the  overall 
results,  with  the  24  km  difference  between  the  lowest  predicted  hmF2  (IRI-90)  and 
the  largest  predicted  hmF2  (CTIPe). 

4.5.1  Electron  Density. 

All  of  the  models  produced  better  results  during  these  moderate  geomagnetic 
storming  conditions  than  the  overall  set  of  results,  with  GAIM  2  and  3  obtaining 
the  highest  correlation  values  (0.941)  of  any  model  during  any  period  (Table  24). 
CTIPe  and  IFM  performed  the  worst  overall,  but  all  of  the  models  showed  very  good 
correlation  overall.  This  relative  increase  in  performance  suggests  the  models  may 
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Table  23.  Run  3  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

284.45 

5.23 

15.00 

IRI-12 

278.48 

4.32 

12.00 

CTIPe 

308.61 

2.35 

5.99 

IFM 

293.05 

5.72 

13.44 

GAIM 

2 

286.53 

5.75 

13.90 

GAIM 

3 

287.02 

5.75 

13.90 

GAIM 

3 

3-hr 

290.21 

5.68 

13.64 

GAIM 

3 

6-hr 

292.30 

5.70 

13.66 

GAIM 

3 

12-hr 

293.01 

5.74 

13.65 

GAIM 

3 

24-hr 

293.04 

5.71 

13.55 

perform  better  during  geomagnetic  storming,  during  periods  when  the  F10.7  index 
correlates  to  EUV  flux,  or  both. 

Table  24.  Run  3  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.869 

59 

27 

IRI-12 

0.882 

41 

-12 

CTIPe 

0.792 

61 

-56 

IFM 

0.799 

93 

49 

GAIM 

2 

0.941 

45 

14 

GAIM 

3 

0.941 

44 

14 

GAIM 

3 

3-hr 

0.900 

59 

26 

GAIM 

3 

6-hr 

0.892 

68 

34 

GAIM 

3 

12-hr 

0.885 

72 

38 

GAIM 

3 

24-hr 

0.886 

72 

38 

Day  time  performance  during  this  period  was  similar  to  the  overall  daytime  results 
and  very  good  in  terms  of  correlation  (Table  25).  At  night,  all  of  the  models  exhibited 
weaker  correlation  than  in  the  overall  nighttime  results.  CTIPe  performed  the  worst 
by  far,  while  GAIM  2  and  3  were  the  most  accurate  at  night.  MPD  values  were 
also  lower  than  the  overall  nighttime  results,  likely  due  to  fewer  plasma  depletions 
measured  by  the  FPMU  during  this  run  (based  on  examining  the  electron  density 
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plots). 


Table  25.  Diurnal  Run  3  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.844 

60 

33 

0.438 

56 

21 

IRI-12 

0.858 

36 

-6 

0.428 

46 

-19 

CTIPe 

0.746 

59 

-59 

0.406 

63 

-51 

IFM 

0.756 

96 

52 

0.475 

89 

45 

GAIM  2 

0.930 

34 

0 

0.756 

58 

30 

GAIM  3 

0.929 

35 

3 

0.764 

55 

26 

GAIM  3  3-hr 

0.878 

56 

22 

0.621 

64 

30 

GAIM  3  6-hr 

0.871 

68 

33 

0.592 

69 

35 

GAIM  3  12-hr 

0.862 

72 

37 

0.572 

73 

41 

GAIM  3  24-hr 

0.863 

70 

34 

0.568 

73 

43 

All  of  the  models  also  exhibited  better  performance  at  low  latitudes  than  in  the 
overall  results,  as  well  as  a  general  decrease  in  MPD,  again  likely  due  to  fewer  equa¬ 
torial  plasma  bubbles  (Table  26).  IFM  was  particularly  bad,  with  a  correlation  of 
only  0.242.  FPMU  encountered  some  mid-latitude  plasma  depletions,  likely  causing 
this  decrease  in  performance. 

Table  26.  Regional  Run  3  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Low  Latitude  (  A  <  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.860 

51 

6 

0.564 

65 

45 

IRI-12 

0.866 

46 

-23 

0.615 

36 

-3 

CTIPe 

0.728 

63 

-53 

0.752 

59 

-58 

IFM 

0.815 

76 

10 

0.242 

107 

82 

GAIM  2 

0.931 

49 

17 

0.795 

42 

11 

GAIM  3 

0.930 

46 

14 

0.789 

43 

13 

GAIM  3  3-hr 

0.890 

58 

14 

0.539 

60 

35 

GAIM  3  6-hr 

0.887 

60 

13 

0.465 

76 

53 

GAIM  3  12-hr 

0.882 

62 

13 

0.426 

81 

60 

GAIM  3  24-hr 

0.882 

63 

15 

0.436 

80 

58 

63 


4.5.2  Electron  Temperature. 


Electron  temperature  performance  during  this  run  was  essentially  the  same  as  for 
the  overall  set  of  data  (Table  27).  IRI-90  again  was  the  most  accurate,  while  IFM 
performed  the  worst. 

Table  27.  Run  3  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.728 

24 

-23 

IRI-12 

0.680 

30 

-29 

CTIPe 

0.574 

22 

1 

IFM 

0.299 

48 

-46 

However,  during  the  daytime  both  IRI  models  performed  better  than  the  overall 

daytime  results  and  much  better  than  Runs  1  and  2  (during  extreme  solar  minimum), 

as  seen  in  Table  28.  CTIPe  and  IFM  had  worse  performance  during  the  day  than 

in  the  overall  results,  although  even  though  the  overall  daytime  performance  values 

were  already  poor.  IFM  again  had  a  negative  correlation  value  (-0.208),  indicating  a 

weak  inverse  linear  relationship.  Night  performance  was  very  similar  to  the  overall 

nighttime  results,  with  slightly  higher  MAPD  values  and  more  of  a  negative  bias. 

Table  28.  Run  3  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.692 

15 

-14 

0.536 

37 

-37 

IRI-12 

0.497 

27 

-27 

0.510 

35 

-34 

CTIPe 

0.129 

18 

10 

0.327 

27 

-9 

IFM 

-0.208 

50 

-50 

0.353 

46 

-40 

At  low-latitudes,  all  of  the  models  performed  similar  to  the  overall  low-latitude 
results,  while  there  was  a  slight  decrease  in  performance  at  mid-latitudes,  with  all 
models  tending  to  show  more  of  a  negative  bias  than  in  the  overall  results  (Table  29). 
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IFM’s  difficulty  in  capturing  electron  temperature  trends  at  mid  latitudes  is  clearly 
evident  in  its  extremely  weak  correlation  value  of  0.093. 

Table  29.  Run  3  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Low  Latitude  (  A 

<  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.674 

31 

-31 

0.518 

21 

-20 

IRI-12 

0.671 

28 

-27 

0.420 

31 

-30 

CTIPe 

0.551 

26 

9 

0.329 

20 

-3 

IFM 

0.656 

38 

-38 

0.093 

53 

-49 

4.6  Run  4  Results  -  High  Solar  Activity  and  Moderate  Geomagnetic 
Storming 

Run  4  represented  moderate  solar  and  geomagnetic  activity  and  occurred  during 
1-5  March  2011,  with  Kp  index  values  peaking  at  5+  on  day  60  and  then  decreasing 
down  to  2  by  the  end  of  day  64.  F10.7  index  values  steadily  increased  with  each  day 
(Figure  16). 

Average  hrriF2,  NrnF2,  and  TEC  values  for  the  period  are  presented  in  Table 
30.  As  expected,  elevated  NmF2  and  TEC  values  are  predicted  during  this  period 
compared  to  the  the  previous  runs  due  to  increased  geomagnetic  and  solar  activity. 
As  a  result,  we  also  see  higher  hmF2  values  than  in  the  previous  runs,  since  higher 
electron  densities  tend  to  shift  the  F2  peak  upward.  The  same  trends  within  the 
models  held,  with  CTIPe  predicted  the  smallest  NmF2  and  TEC  values  and  IFM 
predicting  the  largest  NmF2  and  TEC  values. 

4.6.1  Electron  Density. 

All  of  the  models  except  IRI  performed  worse  during  this  set  of  conditions  than 
in  the  overall  results  (Table  31).  Both  IRI  models  showed  good  performance  overall, 
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Figure  16.  Run  4  Kp  and  F10.7  Index  Values  for  1-5  March  2011  ( sfu  =  10  22W  m  2  s). 


Table  30.  Run  4  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

291.99 

5.01 

14.88 

IRI-12 

291.97 

4.74 

13.47 

CTIPe 

306.12 

2.86 

8.14 

IFM 

305.05 

7.00 

18.80 

GAIM  2 

301.82 

5.54 

15.75 

GAIM  3 

302.10 

5.71 

15.62 

GAIM  3 

3-hr 

303.11 

5.80 

16.34 

GAIM  3 

6-hr 

304.15 

5.89 

16.42 

GAIM  3 

12-hr 

305.22 

5.95 

16.67 

GAIM  3 

24-hr 

305.08 

5.94 

16.04 
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while  all  of  the  other  models  showed  relatively  weak  correlation.  IFM  and  GAIM  had 
very  high  MAPD  and  MPD  values,  with  IFM’s  severe  positive  bias  (two  times  larger 
than  the  overall  results)  likely  causing  GAIM’s  Kalman  filter  to  discard  many  data 
assimilation  sources  due  to  differences  between  the  background  and  the  data  source 
deemed  too  large  and  discarded.  IFM’s  performance  issues  were  likely  related  to  the 
increase  in  solar  activity  across  the  period,  since  it  didn’t  seem  to  struggle  with  the 
decreasing  geomagnetic  activity  levels  seen  in  Run  3. 

Table  31.  Run  4  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.845 

50 

34 

IRI-12 

0.840 

42 

21 

CTIPe 

0.580 

53 

-25 

IFM 

0.527 

146 

128 

GAIM  2 

0.589 

95 

73 

GAIM  3 

0.590 

97 

76 

GAIM  3  3-hr 

0.561 

107 

87 

GAIM  3  6-hr 

0.541 

110 

91 

GAIM  3  12-hr 

0.542 

113 

95 

GAIM  3  24-hr 

0.548 

110 

92 

As  expected,  degraded  performance  overall  was  also  seen  when  looking  at  diurnal 
trends  (Table  32).  All  of  the  models  tended  to  overestimate  more  than  in  the  overall 
results  during  the  day  time.  The  biggest  performance  change  occurred  at  night, 
when  IFM  and  GAIM  both  overestimated  values  by  at  least  121%.  Again  this  is 
tied  to  GAIM’s  reliance  on  IFM  as  a  background;  the  greater  the  differences  in  IFM 
background  and  assimilated  data,  the  less  GAIM  trusts  the  data  source,  leading  to  a 
closer  relationship  between  IFM  and  GAIM  performance. 

At  low- latitudes,  both  versions  of  IRI  again  showed  strong  performance,  while 
the  rest  of  the  models  performed  worse  than  in  the  overall  low-latitude  results  (Table 
33).  GAIM’s  reliance  on  IFM  was  clearly  evident  in  the  performance  data,  with  IFM’s 
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Table  32.  Diurnal  Run  4  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.881 

52 

41 

0.620 

48 

29 

IRI-12 

0.874 

44 

28 

0.641 

40 

15 

CTIPe 

0.610 

48 

-45 

0.516 

57 

-8 

IFM 

0.597 

96 

68 

0.484 

188 

17® 

GAIM  2 

0.680 

53 

15 

0.535 

129 

121 

GAIM  3 

0.673 

55 

18 

0.550 

132 

125 

GAIM  3  3-hr 

0.641 

63 

28 

0.523 

144 

135 

GAIM  3  6-hr 

0.625 

66 

32 

0.510 

147 

139 

GAIM  3  12-hr 

0.627 

68 

36 

0.506 

151 

144 

GAIM  3  24-hr 

0.635 

65 

33 

0.508 

147 

140 

large  MAPD  and  MPD  values  in  both  regions  leading  to  a  closer  relationship  between 
GAIM  and  IFM  performance.  At  mid-latitudes,  IFM  was  completely  uncorrelated  to 
the  FPMU  data,  leading  to  extremely  weak  GAIM  correlation  values  (0.189).  All  of 
the  models  showed  worse  performance  at  mid-latitudes  than  the  overall  mid-latitude 
results,  tending  to  overestimate  more  than  in  the  overall  results,  likely  overcompen¬ 
sating  for  the  increase  in  the  F10  index  values,  which  are  generally  tied  to  an  increase 
in  electron  densities. 


4.6.2  Electron  Temperature. 

Both  IRI  models  performed  very  similarly  during  this  set  of  conditions,  while 
CTIPe  and  IFM  showed  much  worse  correlation  than  in  the  overall  results  (Table 
34).  In  fact,  IFM  was  almost  completely  uncorrelated  during  the  period  (0.073). 

Day  time  performance  overall  was  very  similar  as  well;  CTIPe  and  IRI-12  had 
slightly  stronger  day  time  correlation  values  than  the  overall  results  (but  still  weak 
overall),  while  IRI-90  and  IFM  both  had  weaker  correlation  values  than  the  overall 
results,  with  IFM  again  obtaining  a  negative  correlation  value  (Table  35).  At  night, 


Table  33.  Regional  Run  4  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.804 

43 

17 

0.643  1 

55 

47 

IRI-12 

0.795 

37 

4 

0.635  j 

45 

32 

CTIPe 

0.375 

61 

-9 

0.504 

47 

-36 

IFM 

0.369 

143 

113 

0.000 

148 

138 

GAIM  2 

0.404 

113 

85 

0.189 

82 

65 

GAIM  3 

0.405 

116 

89 

0.181 

84 

67 

GAIM  3  3-hr 

0.364 

126 

97 

0.180 

93 

80 

GAIM  3  6-hr 

0.344 

125 

94 

0.145 

100 

89 

GAIM  3  12-hr 

0.348 

124 

94 

0.140 

106 

96 

GAIM  3  24-hr 

0.353 

122 

92 

0.153 

102 

91 

Table  34.  Run  4  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.681 

32 

-30 

IRI-12 

0.694 

25 

-22 

CTIPe 

0.237 

35 

16 

IFM 

0.073 

41 

-36 
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both  IFM  and  CTIPe  were  effectively  uncorrelated  (both  with  small,  negative  values). 

CTIPe  also  had  a  much  larger  positive  bias  than  in  the  overall  results. 

Table  35.  Run  4  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.297 

20 

-17 

0.609 

41 

-40 

IRI-12 

0.399 

19 

-15 

0.550 

30 

-28 

CTIPe 

0.292 

18 

-5 

-0.054 

46 

30 

IFM 

-0.040 

50 

-48 

-0.085 

34 

-26 

Low-latitude  performance  was  worse  than  the  overall  results  across  the  board 
during  this  period,  with  CTIPe  and  IFM  almost  completely  uncorrelated  (Table  36). 
At  mid-latitudes,  both  IRI  models  performed  better  than  the  overall  results,  while  the 
two  physics-based  models  performed  worse,  again  with  IFM  data  values  completely 
uncorrelated  to  FPMU  values. 

Table  36.  Run  4  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Low  Latitude  (  A  <  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.448 

39 

-39 

0.650 

29 

-27 

IRI-12 

0.491 

32 

-31 

0.657 

23 

-20 

CTIPe 

-0.071 

46 

29 

0.234 

31 

11 

IFM 

0.020 

31 

-22 

0.050 

44 

-40 

4.7  Run  5  Results  -  Moderate  Solar  activity  and  Moderate  Geomagnetic 
Storming 

Run  5  occurred  during  30  April  -  5  May  2011  and  represented  moderate  solar 
and  geomagnetic  activity.  Kp  index  values  peaked  at  5-  on  day  120  and  decreased 
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Figure  17.  Run  5  Kp  and  F10.7  Index  Values  for  30  April  -  5  May  2011  ( sfu  = 

10~22VF  to-2  s). 

throughout  the  period,  while  F10  index  values  remained  relatively  constant  around 
110  (Figure  17). 

Average  hmF2 ,  NmF2,  and  TEC  values  for  the  period  are  presented  in  Table 
37.  Due  to  the  increased  solar  and  geomagnetic  activity,  hmF2,  NmF2,  and  TEC 
values  are  all  higher  in  general  compared  to  the  overall  results  and  the  previous  runs. 
CTIPe  again  predicted  the  highest  hrnF2 ,  and  the  lowest  NmF2  and  TEC  values. 
IRI-12  predicted  the  lowest  hmF2,  GAIM  2  predicted  the  largest  NmF2,  and  GAIM 
3  predicted  the  largest  TEC  values. 

4.7.1  Electron  Density. 

Model  performance  during  this  period  was  very  good  overall,  and  better  than  the 
overall  results  with  the  exception  of  CTIPe  (Table  38).  All  of  the  GAIM  forecasts  also 
performed  better  than  both  IRI  models  and  CTIPe  during  the  period,  likely  due  to 
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Table  37.  Run  5  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

299.77 

5.89 

18.16 

IRI-12 

294.57 

5.29 

15.54 

CTIPe 

308.41 

3.10 

9.23 

IFM 

304.70 

6.67 

17.05 

GAIM 

2 

303.99 

7.15 

19.09 

GAIM 

3 

304.39 

6.99 

19.16 

GAIM 

3 

3-hr 

305.56 

7.00 

18.72 

GAIM 

3 

6-hr 

306.43 

7.06 

18.66 

GAIM 

3 

12-hr 

306.86 

7.08 

18.57 

GAIM 

3 

24-hr 

306.79 

7.08 

18.78 

IFM’s  improved  performance.  CTIPe  had  weaker  correlation  than  the  overall  results, 

and  tended  to  underestimate  electron  densities  more  during  this  period. 

Table  38.  Run  5  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.850 

56 

19 

IRI-12 

0.841 

47 

-2 

CTIPe 

0.686 

57 

-46 

IFM 

0.790 

66 

25 

GAIM  2 

0.912 

46 

18 

GAIM  3 

0.914 

45 

16 

GAIM  3  3-hr 

0.883 

53 

23 

GAIM  3  6-hr 

0.879 

56 

27 

GAIM  3  12-hr 

0.877 

57 

29 

GAIM  3  24-hr 

0.879 

58 

30 

During  the  day,  all  of  models  except  CTIPe  also  performed  better  than  the  overall 
daytime  results  (Table  39).  GAIM  2  and  3  had  very  strong  correlation  to  FPMU 
values,  and  all  of  the  models  except  CTIPe  showed  very  little  overall  bias.  Again  the 
effect  of  IFM’s  improved  performance  was  clearly  evident  in  the  GAIM  models,  with 
all  of  the  GAIM  runs  exhibiting  very  strong  correlation.  Night  time  performance 
during  this  period  was  slightly  better  than  overall  night  performance.  All  of  the 
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models  also  displayed  a  decrease  in  MPD  values  across  the  board. 


Table  39.  Diurnal  Run  5  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.890 

44 

10 

0.761 

67 

28 

IRI-12 

0.893 

37 

-8 

0.761 

57 

4 

CTIPe 

0.696 

53 

-50 

0.602 

61 

-43 

IFM 

0.849 

46 

4 

0.663 

86 

44 

GAIM  2 

0.935 

29 

-3 

0.850 

63 

39 

GAIM  3 

0.936 

29 

-3 

0.853 

60 

36 

GAIM  3  3-hr 

0.915 

37 

5 

0.791 

69 

41 

GAIM  3  6-hr 

0.915 

39 

8 

0.779 

73 

46 

GAIM  3  12-hr 

0.913 

40 

9 

0.781 

74 

49 

GAIM  3  24-hr 

0.914 

41 

10 

0.782 

74 

49 

Regionally,  CTIPe  performed  worse  than  the  overall  results  at  low  latitudes  (Table 
40),  tending  to  severely  under  predicting  equatorial  electron  densities  at  night.  All 
of  the  other  models  showed  a  decrease  in  MPD  values  as  well,  also  due  to  under 
predicting  the  high  electron  densities  experienced  by  the  FPMU  at  night  near  the 
equator.  Mid  latitude  performance  was  slightly  better  than  the  overall  results  across 
the  board. 

4.7.2  Electron  Temperature. 

Overall  electron  temperature  performance  during  this  period  was  slightly  worse 
than  the  overall  results,  but  mainly  only  in  the  form  of  weaker  correlation  values 
(Table  41).  During  the  day,  CTIPe  performed  significantly  worse  than  the  overall 
daytime  results,  obtaining  a  negative  correlation  value  (Table  42).  At  night,  all  of 
the  models  underestimated  more  than  in  the  overall  night  time  results  and  tended 
to  under  predict  electron  temperatures  more.  While  low-latitude  performance  was 
mostly  similar  to  the  overall  results,  all  of  the  models  performed  worse  than  the 
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Table  40.  Regional  Run  5  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.778 

65 

19 

0.833 

49 

20 

IRI-12 

0.771 

57 

-1 

0.833 

40 

-2 

CTIPe 

0.527 

61 

-40 

0.734 

53 

-51 

IFM 

0.719 

73 

13 

0.720 

61 

34 

GAIM  2 

0.866 

57 

26 

0.923 

38 

12 

GAIM  3 

0.870 

55 

25 

0.925 

37 

10 

GAIM  3  3-hr 

0.833 

60 

26 

0.853 

47 

21 

GAIM  3  6-hr 

0.830 

62 

27 

0.837 

51 

28 

GAIM  3  12-hr 

0.828 

63 

27 

0.835 

53 

31 

GAIM  3  24-hr 

0.830 

63 

27 

0.836 

53 

31 

overall  results  at  mid- latitudes,  especially  IFM  (Table  43).  IFM  tended  to  under 
predict  electron  temperatures  more  than  in  the  overall  results  at  mid-latitudes,  in 
spite  of  the  fact  it  severely  over  predicted  electron  temperatures  (sometimes  by  over 
2000  K)  on  several  occasions  at  night  around  —50°  latitude. 

Table  41.  Run  5  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.665 

27 

-26 

IRI-12 

0.684 

25 

-24 

CTIPe 

0.481 

27 

0 

IFM 

0.284 

44 

-43 

4.8  Run  6  Results  -  High  Solar  Activity  and  Low  Geomagnetic  Activity 

Run  6  occurred  during  19-23  October  2011  and  represented  high  solar  activity 
(peaking  at  a  F10.7  value  of  164  on  day  294)  and  a  Kp  index  value  of  2+  (Figure  18). 

Average  hmiT,  NmF2,  and  TEC  values  for  the  period  are  presented  in  Table 
44,  and  show  some  distinct  differences  in  relation  to  the  overall  averages.  The  effect 
of  increased  solar  activity  on  electron  densities  is  clearly  evident,  with  the  highest 
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Table  42.  Run  5  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.349 

17 

-13 

0.492 

37 

-37 

IRI-12 

0.334 

15 

-13 

0.508 

34 

-33 

CTIPe 

-0.321 

25 

8 

0.158 

27 

-6 

IFM 

0.058 

48 

-48 

0.100 

40 

-38 

Table  43.  Run  5  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Low  Latitude  (  A  <  30°) 

Mid  Latitude  (  A  >  30°) 

Model 

Corr. 

MAPD 

MPD 

Corr. 

MAPD 

MPD 

(r) 

(%) 

(%) 

(r) 

(%) 

(%) 

IRI-90 

0.691 

35 

-35 

0.481 

25 

-22 

IRI-12 

0.693 

31 

-30 

0.490 

22 

-21 

CTIPe 

0.424 

24 

6 

0.281 

28 

-3 

IFM 

0.613 

33 

-33 

0.091 

49 

-46 
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Figure  18.  Run  6  Kp  and  F10.7  Index  Values  for  19-23  October  2011  ( sfu 

10~22VF  to-2  s). 
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average  values  calculated  during  this  run.  The  largest  difference  (30.5  km)  in  between 
the  lowest  (IRI-12)  and  highest  (CTIPe)  hrnF2  values  was  also  observed.  Otherwise 
the  same  general  trends  seen  in  the  overall  results  hold  during  this  run.  CTIPe  again 
predicted  NmF2  and  TEC  values  nearly  half  that  of  the  others,  and  IFM  predicted 
the  largest  TEC  values. 

Table  44.  Run  6  average  model-predicted  hmF2,  NmF2,  and  TEC  values. 


Model 

hmF2 

(km) 

NmF2 

(x  10nm-3) 

TEC 

(x  1016m-2) 

IRI-90 

305.56 

6.91 

21.66 

IRI-12 

300.13 

7.13 

21.42 

CTIPe 

330.60 

4.67 

15.64 

IFM 

326.58 

10.73 

32.46 

GAIM  2 

323.52 

9.15 

29.41 

GAIM  3 

323.79 

9.34 

29.41 

GAIM  3  3-hr 

325.74 

9.65 

30.02 

GAIM  3  6-hr 

327.10 

9.89 

30.60 

GAIM  3  12-hr 

327.26 

9.96 

30.86 

GAIM  3  24-hr 

327.23 

9.95 

30.87 

4.8.1  Electron  Density. 

Electron  density  results  for  this  run  were  better  than  the  overall  results  and  very 
good  in  general,  suggesting  the  models  can  account  for  increased  geomagnetic  or  solar 
activity,  but  not  both  at  the  same  time  (Table  45).  All  of  the  models  tended  to  have  a 
smaller  positive  or  more  negative  bias  during  this  period;  CTIPe  and  both  IRI  models 
exhibited  an  overall  negative  bias,  while  IFM  and  the  GAIM  models  had  a  positive 
bias. 

Daytime  performance  during  this  period  was  better  across  the  board,  although 
all  of  the  models  tended  to  under  predict  more  than  in  the  overall  results  during  the 
day,  with  all  but  IFM  exhibiting  a  negative  bias  (Table  46).  Correlation  weakened 
more  at  night  than  in  the  overall  results,  but  MAPD  values  decreased  across  the 
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Table  45.  Run  6  performance  of  ionospheric  model  predicted  electron  density  values 
in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.903 

38 

-17 

IRI-12 

0.896 

39 

-24 

CTIPe 

0.810 

58 

-47 

IFM 

0.811 

68 

41 

GAIM  2 

0.919 

40 

7 

GAIM  3 

0.916 

40 

8 

GAIM  3  3-hr 

0.901 

48 

18 

GAIM  3  6-hr 

0.893 

51 

24 

GAIM  3  12-hr 

0.893 

52 

26 

GAIM  3  24-hr 

0.893 

52 

27 

board  as  well,  with  lower  MPD  values  for  all  models.  As  seen  in  the  overall  results 
for  this  period,  performance  at  low  and  mid  latitudes  was  better  than  the  overall  low 
and  mid  latitude  results,  and  all  models  again  showed  lower  bias  values  than  in  the 
overall  results  (Table  47). 

Table  46.  Diurnal  Run  6  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.878 

37 

-29 

0.561 

40 

-3 

IRI-12 

0.864 

40 

-33 

0.605 

38 

-14 

CTIPe 

0.800 

60 

-60 

0.347 

56 

-33 

IFM 

0.797 

47 

9 

0.622 

92 

78 

GAIM  2 

0.910 

27 

-16 

0.676 

55 

33 

GAIM  3 

0.904 

27 

-15 

0.674 

56 

34 

GAIM  3  3-hr 

0.887 

31 

-7 

0.633 

66 

47 

GAIM  3  6-hr 

0.876 

34 

-3 

0.641 

70 

55 

GAIM  3  12-hr 

0.880 

33 

-3 

0.637 

74 

59 

GAIM  3  24-hr 

0.881 

33 

-3 

0.636 

74 

60 

77 


Table  47.  Regional  Run  6  performance  of  ionospheric  model  predicted  electron  density 
values  in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.880 

45 

-21 

0.800 

34 

-14 

IRI-12 

0.876 

45 

-29 

0.779 

34 

-20 

CTIPe 

0.764 

56 

-33 

0.686 

59 

-57 

IFM 

0.755 

84 

52 

0.613 

57 

34 

GAIM  2 

0.890 

47 

13 

0.880 

35 

2 

GAIM  3 

0.885 

47 

14 

0.879 

36 

3 

GAIM  3  3-hr 

0.872 

56 

25 

0.809 

41 

14 

GAIM  3  6-hr 

0.862 

60 

31 

0.788 

44 

19 

GAIM  3  12-hr 

0.862 

63 

34 

0.795 

45 

21 

GAIM  3  24-hr 

0.861 

63 

34 

0.799 

45 

21 

4.8.2  Electron  Temperature. 

Electron  temperature  correlation  improved  in  all  models  during  this  run;  all  but 
IFM  showed  strong  correlation,  and  CTIPe  had  the  strongest  correlation  of  all  models 
(Table  48).  All  of  the  models  had  higher  MPD  values  than  in  the  overall  results  as 
well,  which  is  likely  related  to  the  overall  decrease  in  electron  density  MPD  values. 
Day  time  correlation  improved  for  all  but  IFM,  which  again  was  effectively  uncorre¬ 
lated  during  the  daytime.  Night  time  performance  was  better  than  than  the  overall 
results  across  the  board  (Table  49).  The  same  general  trends  seen  in  the  overall  run 
performance  were  also  seen  in  the  regional  breakdown  (Table  50). 

Table  48.  Run  6  performance  of  ionospheric  model  predicted  electron  temperature 
values  in  relation  to  FPMU  data. 


Model 

Correlation  (r) 

MAPD  (%) 

MPD  (%) 

IRI-90 

0.806 

22 

-18 

IRI-12 

0.816 

15 

-8 

CTIPe 

0.824 

31 

30 

IFM 

0.619 

27 

-27 
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Table  49.  Run  6  Diurnal  performance  of  ionospheric  model  predicted  electron  temper¬ 
ature  values  in  relation  to  FPMU  data. 


Model 

Corr. 

(r) 

Day 

MAPD 

(%) 

MPD 

(%) 

Corr. 

(r) 

Night 

MAPD 

(%) 

MPD 

(%) 

IRI-90 

0.644 

16 

-12 

0.561 

25 

-20 

IRI-12 

0.456 

14 

-7 

0.753 

16 

-9 

CTIPe 

0.452 

40 

39 

0.851 

27 

27  | 

IFM 

-0.017 

39 

-39 

0.534 

22 

-22 

Table  50.  Run  6  Regional  performance  of  ionospheric  model  predicted  electron  tem¬ 
perature  values  in  relation  to  FPMU  data. 


Model 

Low  Latitude  (  A  <  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

Mid  Latitude  (  A  >  30°) 

Corr.  MAPD  MPD 

(r)  (%)  (%) 

IRI-90 

0.887 

31 

-31 

0.779 

18 

-12 

IRI-12 

0.798 

21 

-18 

0.808 

13 

-4 

CTIPe 

0.703 

34 

34 

0.909 

28 

27 

IFM 

0.750 

20 

-20 

0.616 

30 

-30 
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V.  Conclusions  and  Recommendations 


5.1  Chapter  Overview 

This  chapter  consists  of  two  main  sections.  First,  a  summary  will  be  provided  of 
the  results  of  comparing  FPMU  electron  densities  to  various  ionospheric  models,  to 
include  recommendations  of  models  for  NASA  mission  planning  use.  Then,  recom¬ 
mendations  for  future  FPMU  and  model  validation  efforts  will  be  presented. 

5.2  Conclusions 

GAIM  and  IRI-12  most  accurately  modeled  the  ISS  plasma  environment  for  the 
dates  considered  in  this  effort.  GAIM  modeled  the  electron  density  trends  the  best, 
while  IRI-12  produced  the  lowest  average  errors  and  the  smallest  overall  bias.  GAIM 
performed  very  well  at  predicting  electron  density  values  due  to  its  data  assimilation 
scheme.  This  was  especially  impressive  considering  GAIM’s  3-D  output  was  the  coars¬ 
est  of  all  of  the  models.  The  addition  of  the  DMSP  data  source  didn’t  significantly 
shift  the  F2  peak  or  have  a  significant  effect  on  performance  for  ISS  orbits,  likely 
due  to  the  altitudes  considered  in  this  effort.  GAIM’s  reliance  on  IFM  background 
values  likely  significantly  impacted  its  performance  during  periods  when  IFM  had  the 
largest  errors  (during  deep  solar  minimum  and  periods  of  simultaneous  moderate  solar 
and  geomagnetic  activity).  As  IFM’s  errors  grow,  the  impact  of  GAIM’s  assimilated 
data  diminishes,  since  assimilated  data  values  that  produce  differences  relative  to  the 
IFM  background  that  are  larger  than  the  uncertainty  associated  with  that  data  are 
discarded  or  have  a  weak  weighting  applied  to  them.  The  two  physics  based  models 
performed  the  worst  overall,  most  often  obtaining  the  weakest  correlation  coefficient 
values  and  highest  error  and  biases.  IRI-12’s  improvements  over  IRI-90  were  clearly 
evident,  with  an  18%  decrease  in  MAPD  values,  likely  due  to  many  more  years  worth 
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of  averaged  data  and  improved  NmF2  and  topside  electron  density  models.  Average 
errors  were  relatively  high  overall  (greater  than  46%).  While  these  high  average  er¬ 
rors  confirm  the  importance  of  performing  a  validation  effort  at  this  altitude,  some 
of  these  errors  could  be  due  to  FPMU  measurement  errors  since  a  robust  validation 
effort  and  error  analysis  has  not  been  performed  to  date  for  the  FPMU. 

These  trends  carried  over  to  electron  temperature  values.  IFM’s  mid-latitude 
electron  temperature  model  performed  extremely  poorly.  The  performance  of  both 
IRI  models  overall  was  very  impressive  across  both  electron  densities  and  electron 
temperatures. 

From  an  ISS  mission  planning  standpoint,  GAIM  and  IRI-12  represent  the  best  op¬ 
tions  for  predicting  ISS  electron  densities.  Additionally,  even  if  GAIM’s  performance 
is  skewed  significantly  by  its  IFM  background,  IFM’s  tendency  to  overestimate  values 
across  all  conditions  actually  provides  more  of  a  worst-case  set  of  charging  conditions, 
since  high-density,  cold  plasma  produces  the  largest  amount  of  ISS  charging  in  gen¬ 
eral.  IFM’s  large  negative  electron  temperature  bias  adds  to  this  effect.  Conversely, 
CTIPe’s  tendency  to  underestimate  electron  densities  and  overestimate  electron  tem¬ 
peratures  indicates  it  would  not  be  a  good  candidate  for  ISS  mission  planning,  since 
it  would  likely  under  predict  the  magnitude  of  ISS  charging.  IRI-12’s  strong  per¬ 
formance  across  both  electron  densities  and  temperatures  (overall  and  during  the 
daytime)  and  its  tendency  to  slightly  overestimate  electron  densities  and  underesti¬ 
mate  electron  temperatures  indicate  it  would  also  be  a  good  solution  for  ISS  mission 
planning.  Additionally,  IRI-12  is  much  more  computationally  efficient,  since  no  inter¬ 
polation  would  have  to  take  place  to  produce  ISS  predicted  values,  and  GAIM  also 
requires  IFM  output  to  run.  Overall,  IRI-12  currently  represents  the  best  choice  as  a 
NASA  real-time  mission  planning  tool. 
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5.3  Recommendations  for  Future  Work 


There  are  many  opportunities  for  future  work  that  complements  this  effort,  both 
from  a  model  validation  standpoint  and  for  providing  NASA  with  a  better  ionospheric 
model  than  what  is  currently  in  use  for  mission  planning. 

One  potential  area  of  follow-on  research  is  the  investigation  of  creating  an  “ensem¬ 
ble”  model  for  predicting  ISS  electron  density  values  comprised  of  some  of  the  models 
used  in  this  effort,  with  weights  driven  by  solar  and  geomagnetic  activity  levels.  With 
the  relatively  strong  correlation  coefficient  values  that  some  of  the  models  showed, 
ensemble  modeling  that  attempts  to  correct  for  some  of  the  biases  and  errors  based 
on  geomagnetic  and  solar  activity  could  provide  for  more  accurate  ISS  ionospheric 
predictions. 

A  more  detailed  understanding  of  geomagnetic  and  solar  activity  effects  on  per¬ 
formance  can  be  obtained  by  examining  a  larger  number  of  days  for  each  set  of 
conditions.  This  would  reduce  the  effect  outlier  data  points  have  on  the  analysis,  and 
provide  a  better  idea  of  “normal”  performance  for  each  set  of  conditions.  It  could 
also  be  useful  to  examine  the  space  weather  events  leading  to  increased  geomagnetic 
storming  levels  (as  measured  by  the  Kp  index)  to  analyze  how  well  the  Kp  index  acts 
as  a  proxy  in  the  ionospheric  models  for  those  events. 

Lastly,  to  build  on  the  results  of  this  effort,  a  similar  study  could  be  performed 
using  the  US  Air  Force  Academy’s  Integrated  Miniaturized  ElectroStatic  Analyzer 
(iMESA)  device  aboard  the  ISS,  which  also  measures  electron  densities  and  electron 
temperatures.  This  could  help  further  validate  the  results  presented  here. 
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Appendix  A.  IRI  Inputs 


1.1  IRI-90 


These  are  the  input  flags  that  were  used  for  running  IRI-90: 


T 

T 

T 

T 

F 

T 

T 

T 

T 

T 

T 

F 


JF ( 1 ) = . TRUE . 
JF  (2)  =T  [F] 

JF  (3)  =T  [F] 

JF  (4)  =T  [F] 

JF  (5)  =T  [F] 

JF (6) =T [F] 
JF(7)=T  [F] 

JF  (8)  =T  [F] 
JF(9)=T [F] 
JF(10)=T [F] 
JF(11)=T [F] 
JF(12)=T [F] 


[.FALSE.]  ELECTRON  DENSITY  IS  [NOT]  CALCULATED 
TEMPERATURES  ARE  [NOT]  CALCULATED 
ION  COMPOSITION  IS  [NOT]  CALCULATED 
BO  FROM  TABLE  [FROM  GULYEAVA  1987] 

F2  PEAK  FROM  CCIR  [FROM  URSI] 

ION  COMP.  STANDARD  [DANIL0V-YAICHNIK0V-1985] 
STAND.  IRI  TOPSIDE  [IRI-79] 

NMF2  PEAK  MODEL  [INPUT  VALUES] 

HMF2  PEAK  MODEL  [INPUT  VALUES] 

TE  MODEL  [TE-NE  MODEL  WITH  NE  INPUT] 

NE  STANDARD  [LAY-FUNCTIONS  VERSION] 

MESSAGE  ARE  WRITTEN  TO  UNIT=12  [=6] 


1.2  IRI-12 


These  are  the  input  flags  that  were  used  for  running  IRI-12: 


.true . 


.false . 


standard  version 


1  Ne  computed 

2  Te,  Ti  computed 

3  Ne  &  Ni  computed 

4  BO  -  Table  option 


Ne  not  computed 
Te,  Ti  not  computed 
Ni  not  computed 
BO  -  other  models  j f (31) 


T 

T 

T 

T 
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5 

f oF2  -  CCIR 

f oF2  -  URSI 

F 

6 

Ni  -  DS-95  &  DY-85 

Ni  -  RBV-10  &  TTS-03 

T 

7 

Ne  -  Tops:  fl0.7<188 

fl0.7  unlimited 

T 

8 

foF2  from  model 

foF2  or  NmF2  -  user  input 

T 

9 

hmF2  from  model 

hmF2  or  M3000F2  -  user  input 

T 

10 

Te  -  Standard 

Te  -  Using  Te/Ne  correlation 

T 

11 

Ne  -  Standard  Profile 

Ne  -  Lay-function  formalism 

T 

12 

Messages  to  unit  6 

to  meesages . text  on  unit  11 

F 

13 

foFl  from  model 

foFl  or  NmFl  -  user  input 

T 

14 

hmFl  from  model 

hmFl  -  user  input  (only  Lay  version) t 

15 

foE  from  model 

foE  or  NmE  -  user  input 

T 

16 

hmE  from  model 

hmE  -  user  input 

T 

17 

Rzl2  from  file 

Rzl2  -  user  input 

T 

18 

IGRF  dip,  magbr,  modip 

old  FIELDG  using  P0G068/10  for  1973 

t 

19 

FI  probability  model 

critical  solar  zenith  angle  (old) 

T 

20 

standard  FI 

standard  FI  plus  L  condition 

T 

21 

ion  drift  computed 

ion  drift  not  computed 

F 

22 

ion  densities  in  \% 

ion  densities  in  m-3 

T 

23 

Te_tops  (Aeros,ISIS) 

Te_topside  (TBT-2011) 

F 

24 

D-region:  IRI-95 

Special:  3  D-region  models 

T 

25 

F107D  from  APF107.DAT 

F107D  user  input  (oarr(41)) 

T 

26 

foF2  storm  model 

no  storm  updating 

T 

27 

IG12  from  file 

IG12  -  user 

T 

28 

spread-F  probability 

not  computed  F 

29 

IRIOl-topside 

new  options  as  def .  by  JF(30) 

F 

30 

IRIOl-topside  corr. 

NeQuick  topside  model  false  F 
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(29,30)  =  (t,t)  IRIold,  (f,t)  IRIcor,  (f,f)  NeQuick,  (t,f)  Gulyaeva 


31 

B0,B1  ABT-2009  BO  Gulyaeva  hO . 5 

T 

32 

F10.7_81  from  file 

PF10.7_81  -  user  input 

(oarr(46)) 

T 

33 

Auroral  boundary  model 

on/off  true/false 

T 

34 

Messages  on 

Messages  off 

F 

35 

foE  storm  model 

no  foE  storm  updating 

T 
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